2024-03-18T15:19:11.275 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T15:19:11.275 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} --name pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/orig.config.yaml 2024-03-18T15:19:11.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T15:19:11.417 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879 branch: wip-batrick-testing-20240314.183243 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7608879' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - mons down, quorum - out of quorum - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKO40oG4Z608L2NhycTdYVxMm/Bb/T6zvFkPByFIkbZmmsLgb9YCPYUWiBfyBxOuWNxSX8BZkx/4DqXlYtw4eqw= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGE9XDqXSfddSSffEy/B4ixPhL8iW/G9+DP92bxOlyu4CpzfZ7SvUSGdesZQLnkxyuvYfvbMS5zkfBRdz5QgFvs= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF7t/4WZwq6Lz36AVJ1CrMzvCrR9FpbE3jfiinrdJovPg2FhFVzpXdPntwY9RJBwXNcC7NCqpaY9ajBuNo5Muus= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_14:19:46 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T15:19:11.418 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa; will attempt to use it 2024-03-18T15:19:11.418 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks 2024-03-18T15:19:11.419 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T15:19:11.420 INFO:teuthology.task.internal:Checking packages... 2024-03-18T15:19:11.443 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-18T15:19:11.443 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T15:19:11.443 INFO:teuthology.packaging:ref: None 2024-03-18T15:19:11.443 INFO:teuthology.packaging:tag: None 2024-03-18T15:19:11.443 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-18T15:19:11.443 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:19:11.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-18T15:19:11.708 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162.g3d78f5ba 2024-03-18T15:19:11.710 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T15:19:11.716 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T15:19:11.716 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T15:19:11.739 INFO:teuthology.task.internal:Saving configuration 2024-03-18T15:19:11.758 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T15:19:11.763 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:19:11.787 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.941704', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKO40oG4Z608L2NhycTdYVxMm/Bb/T6zvFkPByFIkbZmmsLgb9YCPYUWiBfyBxOuWNxSX8BZkx/4DqXlYtw4eqw='} 2024-03-18T15:19:11.809 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.943518', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGE9XDqXSfddSSffEy/B4ixPhL8iW/G9+DP92bxOlyu4CpzfZ7SvUSGdesZQLnkxyuvYfvbMS5zkfBRdz5QgFvs='} 2024-03-18T15:19:11.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.944708', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF7t/4WZwq6Lz36AVJ1CrMzvCrR9FpbE3jfiinrdJovPg2FhFVzpXdPntwY9RJBwXNcC7NCqpaY9ajBuNo5Muus='} 2024-03-18T15:19:11.829 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T15:19:11.835 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-18T15:19:11.835 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-18T15:19:11.835 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-18T15:19:11.835 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T15:19:11.940 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f40d00ab160>, signals=[15]) 2024-03-18T15:19:11.940 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T15:19:11.948 INFO:teuthology.task.internal:Opening connections... 2024-03-18T15:19:11.948 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:19:11.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:19:12.041 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-03-18T15:19:12.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:19:12.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-03-18T15:19:12.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:19:12.215 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T15:19:12.225 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-03-18T15:19:12.243 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-03-18T15:19:12.244 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-03-18T15:19:12.300 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T15:19:12.301 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T15:19:12.302 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-03-18T15:19:12.320 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-03-18T15:19:12.338 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-03-18T15:19:12.338 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T15:19:12.399 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T15:19:12.400 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T15:19:12.400 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-18T15:19:12.420 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-03-18T15:19:12.438 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-03-18T15:19:12.438 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-03-18T15:19:12.495 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-03-18T15:19:12.495 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-18T15:19:12.496 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-18T15:19:12.496 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-03-18T15:19:12.520 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T15:19:12.532 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T15:19:12.537 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T15:19:12.538 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-03-18T15:19:12.540 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-03-18T15:19:12.542 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-03-18T15:19:12.558 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T15:19:12.564 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T15:19:12.564 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-03-18T15:19:12.597 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-03-18T15:19:12.603 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-03-18T15:19:12.641 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T15:19:12.700 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T15:19:12.715 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-03-18T15:19:12.715 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-18T15:19:12.715 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-18T15:19:12.715 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T15:19:12.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:19:12.732 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-18T15:19:12.789 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-361.el9.x86_64 2024-03-18T15:19:12.790 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-361.el9.x86_64 2024-03-18T15:19:12.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-03-18T15:19:14.960 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-18T15:19:16.393 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-03-18T15:19:17.600 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-03-18T15:19:22.576 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-03-18T15:19:26.501 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-18T15:19:28.269 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-03-18T15:19:29.986 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-18T15:19:34.944 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-18T15:19:37.084 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-18T15:19:37.084 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-18T15:19:37.183 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-18T15:19:37.194 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-18T15:19:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-427.el9 baseos 6.0 M 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 5.14.0-427.el9 baseos 21 M 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 5.14.0-427.el9 baseos 39 M 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core x86_64 5.14.0-427.el9 baseos 33 M 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-03-18T15:19:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:19:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 100 M 2024-03-18T15:19:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 123 M 2024-03-18T15:19:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-18T15:19:38.195 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): kernel-5.14.0-427.el9.x86_64.rpm 6.3 MB/s | 6.0 MB 00:00 2024-03-18T15:19:39.212 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): kernel-core-5.14.0-427.el9.x86_64.rpm 11 MB/s | 21 MB 00:01 2024-03-18T15:19:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): kernel-modules-core-5.14.0-427.el9.x86_6 17 MB/s | 33 MB 00:01 2024-03-18T15:19:41.204 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): kernel-modules-5.14.0-427.el9.x86_64.rpm 9.9 MB/s | 39 MB 00:03 2024-03-18T15:19:41.204 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-18T15:19:41.204 INFO:teuthology.orchestra.run.smithi096.stdout:Total 25 MB/s | 100 MB 00:04 2024-03-18T15:19:42.085 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-18T15:19:42.457 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-18T15:19:42.457 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-18T15:19:43.815 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-18T15:19:43.816 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-18T15:19:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-18T15:19:47.419 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-core-5.14.0-427.el9.x86_64 1/4 2024-03-18T15:19:47.555 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T15:19:50.357 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T15:19:50.548 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T15:19:55.271 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T15:19:55.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:29.568 INFO:teuthology.orchestra.run.smithi096.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-18T15:20:29.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:20:29.654 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:30.032 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:30.032 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-427.el9.x86_64 1/4 2024-03-18T15:20:30.032 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-5.14.0-427.el9.x86_64 2/4 2024-03-18T15:20:30.033 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-5.14.0-427.el9.x86_64 3/4 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-core-5.14.0-427.el9.x86_64 4/4 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-427.el9.x86_64 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core-5.14.0-427.el9.x86_64 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-5.14.0-427.el9.x86_64 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core-5.14.0-427.el9.x86_64 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:20:30.478 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-18T15:20:30.970 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T15:20:31.107 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T15:20:31.108 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T15:20:31.108 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-427.el9.x86_64 2024-03-18T15:20:31.108 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T15:20:31.108 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T15:20:31.108 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-18T15:20:31.109 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T15:20:31.109 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-03-18T15:20:31.228 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-427.el9.x86_64 2024-03-18T15:20:31.229 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-427.el9.x86_64 2024-03-18T15:20:31.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.06 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 68.el9 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Group : Unspecified 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 8264254 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-18T15:20:31.263 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : builder@centos.org 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:20:31.264 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-03-18T15:20:31.265 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-18T15:20:31.265 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 5.14.0-427.el9.x86_64 2024-03-18T15:20:31.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-18T15:20:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-03-18T15:20:33.265 INFO:teuthology.orchestra.run.smithi096.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-18T15:20:33.299 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-03-18T15:20:33.301 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-03-18T15:20:33.315 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.Myz5NQfvSf 2024-03-18T15:20:33.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Myz5NQfvSf 2024-03-18T15:20:33.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.Myz5NQfvSf 2024-03-18T15:20:33.521 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.Myz5NQfvSf is 6KB 2024-03-18T15:20:33.569 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.Myz5NQfvSf 2024-03-18T15:20:33.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-03-18T15:20:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-18T15:20:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-18T15:20:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-18T15:20:33.653 INFO:teuthology.orchestra.run.smithi096.stdout:15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64.conf 2024-03-18T15:20:33.654 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-427.el9.x86_64 2024-03-18T15:20:33.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-03-18T15:21:03.802 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-18T15:21:03.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:21:03.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:22.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-18T15:21:31.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:21:31.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:34.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-18T15:21:46.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:21:46.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:49.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-18T15:22:04.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:22:04.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:22:07.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-03-18T15:22:25.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:22:25.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:22:25.720 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-18T15:22:26.081 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:22:26.081 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-427.el9.x86_64"... 2024-03-18T15:22:26.081 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-03-18T15:22:26.096 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-427.el9.x86_64 2024-03-18T15:22:26.096 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-427.el9.x86_64 vs 5.14.0-427.el9.x86_64 2024-03-18T15:22:26.097 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T15:22:26.097 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-18T15:22:27.098 DEBUG:teuthology.parallel:result is None 2024-03-18T15:22:27.098 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T15:22:27.106 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T15:22:27.107 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T15:22:27.110 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T15:22:27.113 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T15:22:27.130 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T15:22:27.137 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T15:22:27.147 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T15:22:27.147 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T15:22:27.167 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T15:22:27.171 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T15:22:27.223 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T15:22:27.230 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T15:22:27.230 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T15:22:27.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:22:27.246 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T15:22:27.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:22:27.263 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T15:22:27.266 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T15:22:27.289 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T15:22:27.325 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.357 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.367 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.368 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.376 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.389 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:27.390 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T15:22:27.402 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T15:22:27.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T15:22:27.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T15:22:27.408 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T15:22:27.439 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T15:22:27.447 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T15:22:27.448 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T15:22:27.477 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T15:22:27.481 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T15:22:27.496 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:27.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:27.637 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:27.724 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:27.769 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:22:27.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T15:22:27.836 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:27.882 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:27.946 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:27.990 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:28.033 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:22:28.033 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T15:22:28.102 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:28.146 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:28.205 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:28.289 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:28.333 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:22:28.333 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T15:22:28.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-03-18T15:22:28.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-03-18T15:22:28.407 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-03-18T15:22:28.448 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:22:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:22:28.487 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:22:28.947 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T15:22:28.955 INFO:teuthology.task.internal:Starting timer... 2024-03-18T15:22:28.955 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T15:22:28.966 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T15:22:28.981 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-18T15:22:28.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-03-18T15:22:29.048 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-03-18T15:22:29.050 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-03-18T15:22:29.121 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-03-18T15:22:29.121 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-03-18T15:22:29.183 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-03-18T15:22:29.185 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T15:22:29.185 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-03-18T15:22:29.211 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-03-18T15:22:29.211 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-03-18T15:22:29.233 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-03-18T15:22:29.233 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-03-18T15:22:29.251 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-03-18T15:22:29.251 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-03-18T15:22:29.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:22:29.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:22:29.281 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:22:29.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:22:29.310 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:22:29.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:22:29.338 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T15:22:29.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-03-18T15:22:29.371 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-03-18T15:22:29.426 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-03-18T15:22:29.461 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T15:22:29.471 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T15:22:29.490 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T15:22:29.491 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T15:27:00.239 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-03-18T15:27:00.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:27:00.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:27:00.338 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-03-18T15:27:00.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-03-18T15:27:00.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-18T15:27:00.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:27:00.533 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-18T15:27:00.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-18T15:27:00.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-18T15:27:00.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:27:00.706 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-03-18T15:27:00.773 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-03-18T15:27:00.774 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T15:27:00.785 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T15:27:00.786 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T15:27:00.786 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:27:00.789 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T15:27:00.789 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:27:00.791 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T15:27:00.791 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:27:00.817 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T15:27:00.821 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T15:27:00.833 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T15:27:00.838 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T15:27:00.843 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-18T15:27:00.859 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-18T15:27:00.862 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-03-18T15:27:00.867 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-03-18T15:27:00.876 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-03-18T15:27:00.881 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-03-18T15:27:00.888 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-03-18T15:27:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T15:27:00.897 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T15:27:00.902 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-03-18T15:27:00.909 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T15:27:00.914 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T15:27:00.918 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-18T15:27:00.934 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-18T15:27:00.958 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:27:00.961 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:27:00.961 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-18T15:27:00.962 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.962 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.962 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.962 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.963 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.967 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.978 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.982 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:27:00.983 INFO:teuthology.run_tasks:Running task install... 2024-03-18T15:27:00.991 DEBUG:teuthology.task.install:project ceph 2024-03-18T15:27:00.991 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-18T15:27:00.992 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-18T15:27:00.992 INFO:teuthology.task.install:Using flavor: default 2024-03-18T15:27:00.999 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T15:27:00.999 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-18T15:27:01.000 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-18T15:27:01.000 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:27:01.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-18T15:27:01.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:27:01.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'tag': None, 'wait_for_package': False} 2024-03-18T15:27:01.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:27:01.180 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-18T15:27:01.180 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-18T15:27:01.184 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-18T15:27:01.184 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-18T15:27:01.253 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/ 2024-03-18T15:27:01.253 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2162.g3d78f5ba 2024-03-18T15:27:01.324 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T15:27:01.324 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:27:01.325 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T15:27:01.327 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T15:27:01.328 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:27:01.328 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T15:27:01.358 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T15:27:01.358 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T15:27:01.359 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T15:27:01.360 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T15:27:01.426 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T15:27:01.428 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T15:27:01.449 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-18T15:27:01.449 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:27:01.449 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-18T15:27:01.477 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-18T15:27:01.477 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-18T15:27:01.506 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T15:27:01.508 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T15:27:01.544 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-18T15:27:01.571 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-03-18T15:27:01.573 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-03-18T15:27:01.574 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-03-18T15:27:01.576 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-03-18T15:27:01.624 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-18T15:27:01.690 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-03-18T15:27:01.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-03-18T15:27:01.807 INFO:teuthology.orchestra.run.smithi193.stdout:74 files removed 2024-03-18T15:27:01.835 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T15:27:01.837 INFO:teuthology.orchestra.run.smithi096.stdout:74 files removed 2024-03-18T15:27:01.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T15:27:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:74 files removed 2024-03-18T15:27:01.951 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-18T15:27:02.710 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 141 kB/s | 83 kB 00:00 2024-03-18T15:27:02.728 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 155 kB/s | 83 kB 00:00 2024-03-18T15:27:02.740 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 168 kB/s | 83 kB 00:00 2024-03-18T15:27:03.218 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-18T15:27:03.229 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-18T15:27:03.248 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-03-18T15:27:03.644 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-03-18T15:27:03.654 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-03-18T15:27:03.690 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-03-18T15:27:04.415 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - AppStream 9.2 MB/s | 6.7 MB 00:00 2024-03-18T15:27:04.593 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.7 MB 00:00 2024-03-18T15:27:04.596 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS-9 - AppStream 7.2 MB/s | 6.7 MB 00:00 2024-03-18T15:27:05.957 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-03-18T15:27:06.160 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2024-03-18T15:27:06.214 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-18T15:27:07.172 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-03-18T15:27:08.354 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-03-18T15:27:08.625 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-03-18T15:27:09.665 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-03-18T15:27:11.287 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-18T15:27:13.030 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 19 MB 00:03 2024-03-18T15:27:15.744 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 6.1 MB 00:00 2024-03-18T15:27:17.086 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 16 kB 00:00 2024-03-18T15:27:17.264 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.1 MB 00:00 2024-03-18T15:27:18.773 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-03-18T15:27:20.264 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 885 kB/s | 6.1 MB 00:07 2024-03-18T15:27:20.593 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-18T15:27:21.779 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 21 MB 00:04 2024-03-18T15:27:21.856 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-03-18T15:27:23.916 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-18T15:27:25.482 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-18T15:27:26.745 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-03-18T15:27:27.936 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:27.936 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:27.948 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T15:27:27.949 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T15:27:27.950 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T15:27:27.951 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:27.953 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:27.955 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T15:27:27.956 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T15:27:28.037 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-18T15:27:28.054 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T15:27:28.055 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-18T15:27:28.056 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T15:27:28.057 INFO:teuthology.orchestra.run.smithi193.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T15:27:28.058 INFO:teuthology.orchestra.run.smithi193.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-18T15:27:28.059 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T15:27:28.060 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T15:27:28.061 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-18T15:27:28.062 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T15:27:28.063 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T15:27:28.064 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T15:27:28.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T15:27:28.066 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout:============================================================================================ 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout:Install 150 Packages 2024-03-18T15:27:28.067 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 24 Packages 2024-03-18T15:27:28.068 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:27:28.068 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 224 M 2024-03-18T15:27:28.068 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-03-18T15:27:28.784 INFO:teuthology.orchestra.run.smithi193.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-03-18T15:27:28.874 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-03-18T15:27:28.967 INFO:teuthology.orchestra.run.smithi193.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 4.7 MB/s | 890 kB 00:00 2024-03-18T15:27:29.018 INFO:teuthology.orchestra.run.smithi193.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 2.8 MB/s | 145 kB 00:00 2024-03-18T15:27:29.210 INFO:teuthology.orchestra.run.smithi193.stdout:(4/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 12 MB/s | 2.4 MB 00:00 2024-03-18T15:27:29.301 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:29.302 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:29.316 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T15:27:29.317 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T15:27:29.319 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T15:27:29.320 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:29.322 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:29.325 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T15:27:29.326 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T15:27:29.359 INFO:teuthology.orchestra.run.smithi193.stdout:(5/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 10 MB/s | 1.5 MB 00:00 2024-03-18T15:27:29.407 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-03-18T15:27:29.422 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-18T15:27:29.422 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-03-18T15:27:29.422 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-18T15:27:29.422 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-18T15:27:29.423 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T15:27:29.424 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-18T15:27:29.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:29.426 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-18T15:27:29.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T15:27:29.428 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T15:27:29.429 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-18T15:27:29.430 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T15:27:29.431 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T15:27:29.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T15:27:29.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T15:27:29.434 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout:Install 150 Packages 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 24 Packages 2024-03-18T15:27:29.435 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:27:29.436 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 224 M 2024-03-18T15:27:29.436 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-03-18T15:27:29.460 INFO:teuthology.orchestra.run.smithi193.stdout:(6/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 6.3 MB/s | 5.4 MB 00:00 2024-03-18T15:27:29.685 INFO:teuthology.orchestra.run.smithi193.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 14 MB/s | 4.7 MB 00:00 2024-03-18T15:27:30.280 INFO:teuthology.orchestra.run.smithi137.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-03-18T15:27:30.460 INFO:teuthology.orchestra.run.smithi193.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 13 MB/s | 9.8 MB 00:00 2024-03-18T15:27:30.510 INFO:teuthology.orchestra.run.smithi193.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 505 kB/s | 25 kB 00:00 2024-03-18T15:27:31.063 INFO:teuthology.orchestra.run.smithi137.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.1 MB/s | 890 kB 00:00 2024-03-18T15:27:31.180 INFO:teuthology.orchestra.run.smithi137.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 1.2 MB/s | 145 kB 00:00 2024-03-18T15:27:31.278 INFO:teuthology.orchestra.run.smithi193.stdout:(10/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 9.3 MB/s | 17 MB 00:01 2024-03-18T15:27:31.376 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:31.377 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-18T15:27:31.388 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-18T15:27:31.389 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-18T15:27:31.390 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-18T15:27:31.391 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:31.393 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-18T15:27:31.394 INFO:teuthology.orchestra.run.smithi193.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 1.9 MB/s | 221 kB 00:00 2024-03-18T15:27:31.395 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-18T15:27:31.396 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-18T15:27:31.474 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 6.6 k 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.4 M 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 890 k 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 145 k 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 1.5 M 2024-03-18T15:27:31.494 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 139 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 1.8 M 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 7.4 M 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 49 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 9.8 M 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 48 M 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 264 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 748 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 221 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 27 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 18 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-18T15:27:31.495 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 32 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 747 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 127 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-18T15:27:31.496 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 162 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 320 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 298 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 99 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 85 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 170 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-18T15:27:31.497 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-18T15:27:31.498 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.4 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 3.1 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-18T15:27:31.499 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 21 M 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 34 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 2.4 M 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 248 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 4.7 M 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 17 M 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2162.g3d78f5ba.el9 ceph-noarch 16 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 25 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-18T15:27:31.500 INFO:teuthology.orchestra.run.smithi096.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 163 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 501 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 5.1 M 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-18T15:27:31.501 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-18T15:27:31.502 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 45 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-2162.g3d78f5ba.el9 ceph 131 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-18T15:27:31.503 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-18T15:27:31.504 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-18T15:27:31.505 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-18T15:27:31.506 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================================ 2024-03-18T15:27:31.507 INFO:teuthology.orchestra.run.smithi096.stdout:Install 150 Packages 2024-03-18T15:27:31.508 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 24 Packages 2024-03-18T15:27:31.508 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:27:31.508 INFO:teuthology.orchestra.run.smithi193.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 270 kB/s | 27 kB 00:00 2024-03-18T15:27:31.509 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 224 M 2024-03-18T15:27:31.509 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-03-18T15:27:31.603 INFO:teuthology.orchestra.run.smithi193.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 293 kB/s | 32 kB 00:00 2024-03-18T15:27:31.905 INFO:teuthology.orchestra.run.smithi137.stdout:(4/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 2.7 MB/s | 5.4 MB 00:02 2024-03-18T15:27:31.928 INFO:teuthology.orchestra.run.smithi193.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 2.2 MB/s | 747 kB 00:00 2024-03-18T15:27:32.022 INFO:teuthology.orchestra.run.smithi137.stdout:(5/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-03-18T15:27:32.054 INFO:teuthology.orchestra.run.smithi193.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.3 MB/s | 163 kB 00:00 2024-03-18T15:27:32.171 INFO:teuthology.orchestra.run.smithi193.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.1 MB/s | 127 kB 00:00 2024-03-18T15:27:32.396 INFO:teuthology.orchestra.run.smithi193.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 2.2 MB/s | 501 kB 00:00 2024-03-18T15:27:32.494 INFO:teuthology.orchestra.run.smithi137.stdout:(6/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-03-18T15:27:32.633 INFO:teuthology.orchestra.run.smithi096.stdout:(1/174): ceph-19.0.0-2162.g3d78f5ba.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-03-18T15:27:33.266 INFO:teuthology.orchestra.run.smithi137.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-03-18T15:27:34.225 INFO:teuthology.orchestra.run.smithi096.stdout:(2/174): ceph-fuse-19.0.0-2162.g3d78f5ba.el9.x8 559 kB/s | 890 kB 00:01 2024-03-18T15:27:34.304 INFO:teuthology.orchestra.run.smithi193.stdout:(18/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 2.7 MB/s | 5.1 MB 00:01 2024-03-18T15:27:34.408 INFO:teuthology.orchestra.run.smithi096.stdout:(3/174): ceph-immutable-object-cache-19.0.0-216 790 kB/s | 145 kB 00:00 2024-03-18T15:27:34.430 INFO:teuthology.orchestra.run.smithi193.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 361 kB/s | 45 kB 00:00 2024-03-18T15:27:34.960 INFO:teuthology.orchestra.run.smithi193.stdout:(20/174): ceph-test-19.0.0-2162.g3d78f5ba.el9.x 11 MB/s | 48 MB 00:04 2024-03-18T15:27:34.980 INFO:teuthology.orchestra.run.smithi193.stdout:(21/174): python3-ceph-common-19.0.0-2162.g3d78 238 kB/s | 131 kB 00:00 2024-03-18T15:27:35.089 INFO:teuthology.orchestra.run.smithi193.stdout:(22/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 1.2 MB/s | 162 kB 00:00 2024-03-18T15:27:35.156 INFO:teuthology.orchestra.run.smithi193.stdout:(23/174): python3-rados-19.0.0-2162.g3d78f5ba.e 1.8 MB/s | 320 kB 00:00 2024-03-18T15:27:35.231 INFO:teuthology.orchestra.run.smithi193.stdout:(24/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 2.1 MB/s | 298 kB 00:00 2024-03-18T15:27:35.256 INFO:teuthology.orchestra.run.smithi193.stdout:(25/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 993 kB/s | 99 kB 00:00 2024-03-18T15:27:35.290 INFO:teuthology.orchestra.run.smithi193.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-18T15:27:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:(8/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el9 4.8 MB/s | 9.8 MB 00:02 2024-03-18T15:27:35.357 INFO:teuthology.orchestra.run.smithi193.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 2.5 MB/s | 170 kB 00:00 2024-03-18T15:27:35.398 INFO:teuthology.orchestra.run.smithi137.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 276 kB/s | 25 kB 00:00 2024-03-18T15:27:35.424 INFO:teuthology.orchestra.run.smithi193.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 511 kB/s | 34 kB 00:00 2024-03-18T15:27:35.549 INFO:teuthology.orchestra.run.smithi193.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.1 MB/s | 139 kB 00:00 2024-03-18T15:27:35.800 INFO:teuthology.orchestra.run.smithi193.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 7.2 MB/s | 1.8 MB 00:00 2024-03-18T15:27:35.825 INFO:teuthology.orchestra.run.smithi096.stdout:(4/174): ceph-common-19.0.0-2162.g3d78f5ba.el9. 5.7 MB/s | 21 MB 00:03 2024-03-18T15:27:36.208 INFO:teuthology.orchestra.run.smithi096.stdout:(5/174): ceph-mgr-19.0.0-2162.g3d78f5ba.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-03-18T15:27:36.667 INFO:teuthology.orchestra.run.smithi096.stdout:(6/174): ceph-base-19.0.0-2162.g3d78f5ba.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-03-18T15:27:36.698 INFO:teuthology.orchestra.run.smithi137.stdout:(10/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 4.0 MB/s | 17 MB 00:04 2024-03-18T15:27:36.720 INFO:teuthology.orchestra.run.smithi193.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-03-18T15:27:36.766 INFO:teuthology.orchestra.run.smithi193.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 5.2 MB/s | 248 kB 00:00 2024-03-18T15:27:36.800 INFO:teuthology.orchestra.run.smithi193.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 1.4 MB/s | 49 kB 00:00 2024-03-18T15:27:36.834 INFO:teuthology.orchestra.run.smithi193.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 495 kB/s | 16 kB 00:00 2024-03-18T15:27:36.884 INFO:teuthology.orchestra.run.smithi193.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 5.1 MB/s | 264 kB 00:00 2024-03-18T15:27:36.898 INFO:teuthology.orchestra.run.smithi137.stdout:(11/174): ceph-common-19.0.0-2162.g3d78f5ba.el9 2.9 MB/s | 21 MB 00:06 2024-03-18T15:27:36.909 INFO:teuthology.orchestra.run.smithi096.stdout:(7/174): ceph-mon-19.0.0-2162.g3d78f5ba.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-03-18T15:27:36.923 INFO:teuthology.orchestra.run.smithi137.stdout:(12/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 985 kB/s | 221 kB 00:00 2024-03-18T15:27:36.934 INFO:teuthology.orchestra.run.smithi193.stdout:(36/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 1.8 MB/s | 3.1 MB 00:01 2024-03-18T15:27:36.949 INFO:teuthology.orchestra.run.smithi137.stdout:(13/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 542 kB/s | 27 kB 00:00 2024-03-18T15:27:36.982 INFO:teuthology.orchestra.run.smithi137.stdout:(14/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 542 kB/s | 32 kB 00:00 2024-03-18T15:27:37.010 INFO:teuthology.orchestra.run.smithi193.stdout:(37/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 5.8 MB/s | 748 kB 00:00 2024-03-18T15:27:37.035 INFO:teuthology.orchestra.run.smithi193.stdout:(38/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 182 kB/s | 18 kB 00:00 2024-03-18T15:27:37.041 INFO:teuthology.orchestra.run.smithi137.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 2.7 MB/s | 163 kB 00:00 2024-03-18T15:27:37.091 INFO:teuthology.orchestra.run.smithi137.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 2.5 MB/s | 127 kB 00:00 2024-03-18T15:27:37.175 INFO:teuthology.orchestra.run.smithi137.stdout:(17/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 3.2 MB/s | 747 kB 00:00 2024-03-18T15:27:37.209 INFO:teuthology.orchestra.run.smithi096.stdout:(8/174): ceph-mds-19.0.0-2162.g3d78f5ba.el9.x86 863 kB/s | 2.4 MB 00:02 2024-03-18T15:27:37.225 INFO:teuthology.orchestra.run.smithi137.stdout:(18/174): libradosstriper1-19.0.0-2162.g3d78f5b 3.7 MB/s | 501 kB 00:00 2024-03-18T15:27:37.267 INFO:teuthology.orchestra.run.smithi137.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 1.1 MB/s | 45 kB 00:00 2024-03-18T15:27:37.318 INFO:teuthology.orchestra.run.smithi096.stdout:(9/174): ceph-selinux-19.0.0-2162.g3d78f5ba.el9 232 kB/s | 25 kB 00:00 2024-03-18T15:27:37.394 INFO:teuthology.orchestra.run.smithi137.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.0 MB/s | 131 kB 00:00 2024-03-18T15:27:37.442 INFO:teuthology.orchestra.run.smithi137.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 3.3 MB/s | 162 kB 00:00 2024-03-18T15:27:37.452 INFO:teuthology.orchestra.run.smithi193.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-03-18T15:27:37.494 INFO:teuthology.orchestra.run.smithi193.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2024-03-18T15:27:37.526 INFO:teuthology.orchestra.run.smithi137.stdout:(22/174): python3-rados-19.0.0-2162.g3d78f5ba.e 3.8 MB/s | 320 kB 00:00 2024-03-18T15:27:37.527 INFO:teuthology.orchestra.run.smithi193.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-03-18T15:27:37.578 INFO:teuthology.orchestra.run.smithi193.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-18T15:27:37.601 INFO:teuthology.orchestra.run.smithi137.stdout:(23/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.9 MB/s | 298 kB 00:00 2024-03-18T15:27:37.603 INFO:teuthology.orchestra.run.smithi193.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 420 kB/s | 32 kB 00:00 2024-03-18T15:27:37.652 INFO:teuthology.orchestra.run.smithi137.stdout:(24/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.9 MB/s | 99 kB 00:00 2024-03-18T15:27:37.702 INFO:teuthology.orchestra.run.smithi137.stdout:(25/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-18T15:27:37.787 INFO:teuthology.orchestra.run.smithi193.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 93 kB/s | 17 kB 00:00 2024-03-18T15:27:37.862 INFO:teuthology.orchestra.run.smithi193.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-18T15:27:37.887 INFO:teuthology.orchestra.run.smithi193.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 10 kB 00:00 2024-03-18T15:27:37.937 INFO:teuthology.orchestra.run.smithi193.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-18T15:27:37.963 INFO:teuthology.orchestra.run.smithi193.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-03-18T15:27:37.988 INFO:teuthology.orchestra.run.smithi193.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-18T15:27:38.038 INFO:teuthology.orchestra.run.smithi193.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-03-18T15:27:38.068 INFO:teuthology.orchestra.run.smithi096.stdout:(10/174): ceph-radosgw-19.0.0-2162.g3d78f5ba.el 8.5 MB/s | 9.8 MB 00:01 2024-03-18T15:27:38.069 INFO:teuthology.orchestra.run.smithi193.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 577 kB/s | 46 kB 00:00 2024-03-18T15:27:38.095 INFO:teuthology.orchestra.run.smithi193.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 282 kB 00:00 2024-03-18T15:27:38.120 INFO:teuthology.orchestra.run.smithi193.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-18T15:27:38.136 INFO:teuthology.orchestra.run.smithi137.stdout:(26/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 7.1 MB/s | 3.1 MB 00:00 2024-03-18T15:27:38.143 INFO:teuthology.orchestra.run.smithi096.stdout:(11/174): cephfs-mirror-19.0.0-2162.g3d78f5ba.e 2.9 MB/s | 221 kB 00:00 2024-03-18T15:27:38.145 INFO:teuthology.orchestra.run.smithi193.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-18T15:27:38.171 INFO:teuthology.orchestra.run.smithi193.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 677 kB/s | 34 kB 00:00 2024-03-18T15:27:38.186 INFO:teuthology.orchestra.run.smithi137.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 3.4 MB/s | 170 kB 00:00 2024-03-18T15:27:38.228 INFO:teuthology.orchestra.run.smithi137.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 817 kB/s | 34 kB 00:00 2024-03-18T15:27:38.243 INFO:teuthology.orchestra.run.smithi096.stdout:(12/174): cephfs-shell-19.0.0-2162.g3d78f5ba.el 271 kB/s | 27 kB 00:00 2024-03-18T15:27:38.312 INFO:teuthology.orchestra.run.smithi137.stdout:(29/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 4.5 MB/s | 5.1 MB 00:01 2024-03-18T15:27:38.335 INFO:teuthology.orchestra.run.smithi096.stdout:(13/174): libcephfs-devel-19.0.0-2162.g3d78f5ba 346 kB/s | 32 kB 00:00 2024-03-18T15:27:38.336 INFO:teuthology.orchestra.run.smithi137.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 1.2 MB/s | 139 kB 00:00 2024-03-18T15:27:38.363 INFO:teuthology.orchestra.run.smithi193.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-03-18T15:27:38.397 INFO:teuthology.orchestra.run.smithi193.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 2024-03-18T15:27:38.430 INFO:teuthology.orchestra.run.smithi193.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-03-18T15:27:38.456 INFO:teuthology.orchestra.run.smithi193.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-03-18T15:27:38.485 INFO:teuthology.orchestra.run.smithi096.stdout:(14/174): libcephfs2-19.0.0-2162.g3d78f5ba.el9. 4.9 MB/s | 747 kB 00:00 2024-03-18T15:27:38.586 INFO:teuthology.orchestra.run.smithi096.stdout:(15/174): libcephsqlite-19.0.0-2162.g3d78f5ba.e 1.6 MB/s | 163 kB 00:00 2024-03-18T15:27:38.654 INFO:teuthology.orchestra.run.smithi137.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 5.2 MB/s | 1.8 MB 00:00 2024-03-18T15:27:38.661 INFO:teuthology.orchestra.run.smithi096.stdout:(16/174): librados-devel-19.0.0-2162.g3d78f5ba. 1.7 MB/s | 127 kB 00:00 2024-03-18T15:27:38.672 INFO:teuthology.orchestra.run.smithi193.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-03-18T15:27:38.706 INFO:teuthology.orchestra.run.smithi193.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.0 MB/s | 1.0 MB 00:00 2024-03-18T15:27:38.723 INFO:teuthology.orchestra.run.smithi193.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-18T15:27:38.745 INFO:teuthology.orchestra.run.smithi137.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 2.7 MB/s | 248 kB 00:00 2024-03-18T15:27:38.795 INFO:teuthology.orchestra.run.smithi096.stdout:(17/174): libradosstriper1-19.0.0-2162.g3d78f5b 3.7 MB/s | 501 kB 00:00 2024-03-18T15:27:38.795 INFO:teuthology.orchestra.run.smithi137.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 992 kB/s | 49 kB 00:00 2024-03-18T15:27:38.879 INFO:teuthology.orchestra.run.smithi137.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 198 kB/s | 16 kB 00:00 2024-03-18T15:27:38.955 INFO:teuthology.orchestra.run.smithi137.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.4 MB/s | 264 kB 00:00 2024-03-18T15:27:38.956 INFO:teuthology.orchestra.run.smithi193.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-18T15:27:38.973 INFO:teuthology.orchestra.run.smithi193.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 97 kB/s | 24 kB 00:00 2024-03-18T15:27:38.990 INFO:teuthology.orchestra.run.smithi193.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 546 kB/s | 18 kB 00:00 2024-03-18T15:27:39.024 INFO:teuthology.orchestra.run.smithi193.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-18T15:27:39.041 INFO:teuthology.orchestra.run.smithi193.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 627 kB/s | 32 kB 00:00 2024-03-18T15:27:39.155 INFO:teuthology.orchestra.run.smithi137.stdout:(36/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 3.7 MB/s | 748 kB 00:00 2024-03-18T15:27:39.205 INFO:teuthology.orchestra.run.smithi137.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 366 kB/s | 18 kB 00:00 2024-03-18T15:27:39.241 INFO:teuthology.orchestra.run.smithi193.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-03-18T15:27:39.266 INFO:teuthology.orchestra.run.smithi193.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 967 kB/s | 217 kB 00:00 2024-03-18T15:27:39.292 INFO:teuthology.orchestra.run.smithi193.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-18T15:27:39.317 INFO:teuthology.orchestra.run.smithi193.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-18T15:27:39.334 INFO:teuthology.orchestra.run.smithi193.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 987 kB/s | 41 kB 00:00 2024-03-18T15:27:39.384 INFO:teuthology.orchestra.run.smithi193.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-18T15:27:39.418 INFO:teuthology.orchestra.run.smithi193.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-18T15:27:39.430 INFO:teuthology.orchestra.run.smithi137.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-03-18T15:27:39.452 INFO:teuthology.orchestra.run.smithi193.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-18T15:27:39.469 INFO:teuthology.orchestra.run.smithi193.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2024-03-18T15:27:39.494 INFO:teuthology.orchestra.run.smithi193.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-18T15:27:39.547 INFO:teuthology.orchestra.run.smithi137.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2024-03-18T15:27:39.587 INFO:teuthology.orchestra.run.smithi096.stdout:(18/174): librgw2-19.0.0-2162.g3d78f5ba.el9.x86 6.5 MB/s | 5.1 MB 00:00 2024-03-18T15:27:39.670 INFO:teuthology.orchestra.run.smithi096.stdout:(19/174): python3-ceph-argparse-19.0.0-2162.g3d 544 kB/s | 45 kB 00:00 2024-03-18T15:27:39.722 INFO:teuthology.orchestra.run.smithi137.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 920 kB 00:00 2024-03-18T15:27:39.745 INFO:teuthology.orchestra.run.smithi096.stdout:(20/174): python3-ceph-common-19.0.0-2162.g3d78 1.7 MB/s | 131 kB 00:00 2024-03-18T15:27:39.748 INFO:teuthology.orchestra.run.smithi137.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 530 kB/s | 106 kB 00:00 2024-03-18T15:27:39.761 INFO:teuthology.orchestra.run.smithi193.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 145 kB/s | 39 kB 00:00 2024-03-18T15:27:39.806 INFO:teuthology.orchestra.run.smithi137.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-18T15:27:39.812 INFO:teuthology.orchestra.run.smithi096.stdout:(21/174): python3-cephfs-19.0.0-2162.g3d78f5ba. 2.4 MB/s | 162 kB 00:00 2024-03-18T15:27:39.832 INFO:teuthology.orchestra.run.smithi137.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 380 kB/s | 32 kB 00:00 2024-03-18T15:27:39.979 INFO:teuthology.orchestra.run.smithi096.stdout:(22/174): ceph-osd-19.0.0-2162.g3d78f5ba.el9.x8 5.1 MB/s | 17 MB 00:03 2024-03-18T15:27:40.005 INFO:teuthology.orchestra.run.smithi096.stdout:(23/174): python3-rados-19.0.0-2162.g3d78f5ba.e 1.6 MB/s | 320 kB 00:00 2024-03-18T15:27:40.007 INFO:teuthology.orchestra.run.smithi137.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-03-18T15:27:40.053 INFO:teuthology.orchestra.run.smithi193.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-03-18T15:27:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:(24/174): python3-rbd-19.0.0-2162.g3d78f5ba.el9 3.9 MB/s | 298 kB 00:00 2024-03-18T15:27:40.070 INFO:teuthology.orchestra.run.smithi193.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 133 kB/s | 41 kB 00:00 2024-03-18T15:27:40.080 INFO:teuthology.orchestra.run.smithi096.stdout:(25/174): python3-rgw-19.0.0-2162.g3d78f5ba.el9 1.3 MB/s | 99 kB 00:00 2024-03-18T15:27:40.095 INFO:teuthology.orchestra.run.smithi193.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-03-18T15:27:40.105 INFO:teuthology.orchestra.run.smithi096.stdout:(26/174): rbd-fuse-19.0.0-2162.g3d78f5ba.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-18T15:27:40.112 INFO:teuthology.orchestra.run.smithi193.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-03-18T15:27:40.129 INFO:teuthology.orchestra.run.smithi193.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-03-18T15:27:40.146 INFO:teuthology.orchestra.run.smithi193.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-18T15:27:40.156 INFO:teuthology.orchestra.run.smithi096.stdout:(27/174): rbd-nbd-19.0.0-2162.g3d78f5ba.el9.x86 3.3 MB/s | 170 kB 00:00 2024-03-18T15:27:40.157 INFO:teuthology.orchestra.run.smithi137.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-03-18T15:27:40.171 INFO:teuthology.orchestra.run.smithi193.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-03-18T15:27:40.182 INFO:teuthology.orchestra.run.smithi137.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 10 kB 00:00 2024-03-18T15:27:40.189 INFO:teuthology.orchestra.run.smithi096.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2162.g 1.0 MB/s | 34 kB 00:00 2024-03-18T15:27:40.196 INFO:teuthology.orchestra.run.smithi193.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-18T15:27:40.216 INFO:teuthology.orchestra.run.smithi137.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 159 kB 00:00 2024-03-18T15:27:40.231 INFO:teuthology.orchestra.run.smithi096.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2162.g3d78f5b 3.3 MB/s | 139 kB 00:00 2024-03-18T15:27:40.242 INFO:teuthology.orchestra.run.smithi137.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2024-03-18T15:27:40.247 INFO:teuthology.orchestra.run.smithi193.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.1 MB 00:00 2024-03-18T15:27:40.264 INFO:teuthology.orchestra.run.smithi193.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-03-18T15:27:40.289 INFO:teuthology.orchestra.run.smithi193.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.9 MB/s | 124 kB 00:00 2024-03-18T15:27:40.292 INFO:teuthology.orchestra.run.smithi137.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-18T15:27:40.314 INFO:teuthology.orchestra.run.smithi193.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-18T15:27:40.334 INFO:teuthology.orchestra.run.smithi137.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-18T15:27:40.359 INFO:teuthology.orchestra.run.smithi137.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 703 kB/s | 46 kB 00:00 2024-03-18T15:27:40.393 INFO:teuthology.orchestra.run.smithi137.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-03-18T15:27:40.418 INFO:teuthology.orchestra.run.smithi137.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-03-18T15:27:40.443 INFO:teuthology.orchestra.run.smithi137.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-18T15:27:40.456 INFO:teuthology.orchestra.run.smithi096.stdout:(30/174): rbd-mirror-19.0.0-2162.g3d78f5ba.el9. 8.2 MB/s | 3.1 MB 00:00 2024-03-18T15:27:40.469 INFO:teuthology.orchestra.run.smithi137.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 687 kB/s | 34 kB 00:00 2024-03-18T15:27:40.515 INFO:teuthology.orchestra.run.smithi096.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2162.g3d78f 6.3 MB/s | 1.8 MB 00:00 2024-03-18T15:27:40.590 INFO:teuthology.orchestra.run.smithi096.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2162.g3d 3.2 MB/s | 248 kB 00:00 2024-03-18T15:27:40.624 INFO:teuthology.orchestra.run.smithi096.stdout:(33/174): ceph-mgr-rook-19.0.0-2162.g3d78f5ba.e 1.5 MB/s | 49 kB 00:00 2024-03-18T15:27:40.657 INFO:teuthology.orchestra.run.smithi096.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2162.g3 494 kB/s | 16 kB 00:00 2024-03-18T15:27:40.733 INFO:teuthology.orchestra.run.smithi096.stdout:(35/174): ceph-volume-19.0.0-2162.g3d78f5ba.el9 3.4 MB/s | 264 kB 00:00 2024-03-18T15:27:40.794 INFO:teuthology.orchestra.run.smithi137.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 4.7 MB/s | 1.5 MB 00:00 2024-03-18T15:27:40.861 INFO:teuthology.orchestra.run.smithi137.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-03-18T15:27:40.866 INFO:teuthology.orchestra.run.smithi096.stdout:(36/174): cephadm-19.0.0-2162.g3d78f5ba.el9.noa 5.5 MB/s | 748 kB 00:00 2024-03-18T15:27:40.906 INFO:teuthology.orchestra.run.smithi137.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 7.1 MB/s | 811 kB 00:00 2024-03-18T15:27:40.917 INFO:teuthology.orchestra.run.smithi096.stdout:(37/174): cephfs-top-19.0.0-2162.g3d78f5ba.el9. 366 kB/s | 18 kB 00:00 2024-03-18T15:27:40.931 INFO:teuthology.orchestra.run.smithi137.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2024-03-18T15:27:41.209 INFO:teuthology.orchestra.run.smithi096.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2024-03-18T15:27:41.248 INFO:teuthology.orchestra.run.smithi193.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 195 kB/s | 187 kB 00:00 2024-03-18T15:27:41.282 INFO:teuthology.orchestra.run.smithi137.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2024-03-18T15:27:41.302 INFO:teuthology.orchestra.run.smithi096.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 9.8 MB/s | 920 kB 00:00 2024-03-18T15:27:41.315 INFO:teuthology.orchestra.run.smithi137.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-03-18T15:27:41.342 INFO:teuthology.orchestra.run.smithi096.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-03-18T15:27:41.357 INFO:teuthology.orchestra.run.smithi137.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-03-18T15:27:41.434 INFO:teuthology.orchestra.run.smithi096.stdout:(41/174): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-03-18T15:27:41.460 INFO:teuthology.orchestra.run.smithi096.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 307 kB 00:00 2024-03-18T15:27:41.485 INFO:teuthology.orchestra.run.smithi096.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 640 kB/s | 32 kB 00:00 2024-03-18T15:27:41.585 INFO:teuthology.orchestra.run.smithi096.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 171 kB/s | 17 kB 00:00 2024-03-18T15:27:41.616 INFO:teuthology.orchestra.run.smithi137.stdout:(63/174): python3-jsonpatch-1.21-16.el9.noarch. 95 kB/s | 24 kB 00:00 2024-03-18T15:27:41.636 INFO:teuthology.orchestra.run.smithi096.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-03-18T15:27:41.661 INFO:teuthology.orchestra.run.smithi096.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 138 kB/s | 10 kB 00:00 2024-03-18T15:27:41.686 INFO:teuthology.orchestra.run.smithi096.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-18T15:27:41.691 INFO:teuthology.orchestra.run.smithi137.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2024-03-18T15:27:41.714 INFO:teuthology.orchestra.run.smithi096.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-18T15:27:41.716 INFO:teuthology.orchestra.run.smithi137.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 180 kB/s | 18 kB 00:00 2024-03-18T15:27:41.750 INFO:teuthology.orchestra.run.smithi137.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-18T15:27:41.762 INFO:teuthology.orchestra.run.smithi096.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-03-18T15:27:41.775 INFO:teuthology.orchestra.run.smithi137.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 539 kB/s | 32 kB 00:00 2024-03-18T15:27:41.803 INFO:teuthology.orchestra.run.smithi096.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-03-18T15:27:41.829 INFO:teuthology.orchestra.run.smithi096.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 706 kB/s | 46 kB 00:00 2024-03-18T15:27:41.871 INFO:teuthology.orchestra.run.smithi096.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-03-18T15:27:41.896 INFO:teuthology.orchestra.run.smithi096.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-03-18T15:27:41.938 INFO:teuthology.orchestra.run.smithi096.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-18T15:27:41.955 INFO:teuthology.orchestra.run.smithi096.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 584 kB/s | 34 kB 00:00 2024-03-18T15:27:42.142 INFO:teuthology.orchestra.run.smithi137.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 4.8 MB 00:00 2024-03-18T15:27:42.168 INFO:teuthology.orchestra.run.smithi137.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 555 kB/s | 217 kB 00:00 2024-03-18T15:27:42.189 INFO:teuthology.orchestra.run.smithi096.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-03-18T15:27:42.201 INFO:teuthology.orchestra.run.smithi137.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-03-18T15:27:42.226 INFO:teuthology.orchestra.run.smithi137.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-18T15:27:42.231 INFO:teuthology.orchestra.run.smithi096.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 5.5 MB/s | 1.5 MB 00:00 2024-03-18T15:27:42.252 INFO:teuthology.orchestra.run.smithi137.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 826 kB/s | 41 kB 00:00 2024-03-18T15:27:42.274 INFO:teuthology.orchestra.run.smithi096.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 9.9 MB/s | 811 kB 00:00 2024-03-18T15:27:42.286 INFO:teuthology.orchestra.run.smithi137.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-18T15:27:42.298 INFO:teuthology.orchestra.run.smithi096.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-03-18T15:27:42.298 INFO:teuthology.orchestra.run.smithi193.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 186 kB/s | 195 kB 00:01 2024-03-18T15:27:42.310 INFO:teuthology.orchestra.run.smithi137.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-03-18T15:27:42.353 INFO:teuthology.orchestra.run.smithi137.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-03-18T15:27:42.370 INFO:teuthology.orchestra.run.smithi137.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 805 kB/s | 47 kB 00:00 2024-03-18T15:27:42.403 INFO:teuthology.orchestra.run.smithi137.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-03-18T15:27:42.432 INFO:teuthology.orchestra.run.smithi193.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 379 kB/s | 803 kB 00:02 2024-03-18T15:27:42.540 INFO:teuthology.orchestra.run.smithi096.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-18T15:27:42.574 INFO:teuthology.orchestra.run.smithi193.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 152 kB/s | 21 kB 00:00 2024-03-18T15:27:42.632 INFO:teuthology.orchestra.run.smithi193.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 437 kB/s | 25 kB 00:00 2024-03-18T15:27:42.757 INFO:teuthology.orchestra.run.smithi096.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-03-18T15:27:42.782 INFO:teuthology.orchestra.run.smithi096.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 323 kB/s | 78 kB 00:00 2024-03-18T15:27:42.829 INFO:teuthology.orchestra.run.smithi137.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 91 kB/s | 39 kB 00:00 2024-03-18T15:27:42.999 INFO:teuthology.orchestra.run.smithi096.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2024-03-18T15:27:43.016 INFO:teuthology.orchestra.run.smithi096.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 105 kB/s | 24 kB 00:00 2024-03-18T15:27:43.033 INFO:teuthology.orchestra.run.smithi096.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 544 kB/s | 18 kB 00:00 2024-03-18T15:27:43.041 INFO:teuthology.orchestra.run.smithi193.stdout:(96/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 10 MB/s | 4.1 MB 00:00 2024-03-18T15:27:43.058 INFO:teuthology.orchestra.run.smithi096.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-18T15:27:43.075 INFO:teuthology.orchestra.run.smithi096.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 760 kB/s | 32 kB 00:00 2024-03-18T15:27:43.091 INFO:teuthology.orchestra.run.smithi193.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-03-18T15:27:43.129 INFO:teuthology.orchestra.run.smithi137.stdout:(79/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 137 kB/s | 41 kB 00:00 2024-03-18T15:27:43.158 INFO:teuthology.orchestra.run.smithi193.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-03-18T15:27:43.183 INFO:teuthology.orchestra.run.smithi193.stdout:(99/174): python3-requests-2.25.1-8.el9.noarch. 141 kB/s | 125 kB 00:00 2024-03-18T15:27:43.233 INFO:teuthology.orchestra.run.smithi193.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-03-18T15:27:43.275 INFO:teuthology.orchestra.run.smithi096.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2024-03-18T15:27:43.292 INFO:teuthology.orchestra.run.smithi193.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 763 kB/s | 44 kB 00:00 2024-03-18T15:27:43.301 INFO:teuthology.orchestra.run.smithi096.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 965 kB/s | 217 kB 00:00 2024-03-18T15:27:43.319 INFO:teuthology.orchestra.run.smithi193.stdout:(102/174): qatlib-service-23.11.0-1.el9.x86_64. 258 kB/s | 35 kB 00:00 2024-03-18T15:27:43.326 INFO:teuthology.orchestra.run.smithi096.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-18T15:27:43.344 INFO:teuthology.orchestra.run.smithi193.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-18T15:27:43.354 INFO:teuthology.orchestra.run.smithi096.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-18T15:27:43.377 INFO:teuthology.orchestra.run.smithi096.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 823 kB/s | 41 kB 00:00 2024-03-18T15:27:43.402 INFO:teuthology.orchestra.run.smithi096.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-18T15:27:43.421 INFO:teuthology.orchestra.run.smithi137.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-03-18T15:27:43.427 INFO:teuthology.orchestra.run.smithi096.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-18T15:27:43.446 INFO:teuthology.orchestra.run.smithi137.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 949 kB/s | 300 kB 00:00 2024-03-18T15:27:43.452 INFO:teuthology.orchestra.run.smithi096.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-18T15:27:43.472 INFO:teuthology.orchestra.run.smithi137.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T15:27:43.477 INFO:teuthology.orchestra.run.smithi096.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-18T15:27:43.497 INFO:teuthology.orchestra.run.smithi137.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-18T15:27:43.502 INFO:teuthology.orchestra.run.smithi096.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 852 kB/s | 43 kB 00:00 2024-03-18T15:27:43.522 INFO:teuthology.orchestra.run.smithi137.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2024-03-18T15:27:43.547 INFO:teuthology.orchestra.run.smithi137.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-18T15:27:43.581 INFO:teuthology.orchestra.run.smithi137.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-18T15:27:43.664 INFO:teuthology.orchestra.run.smithi137.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.1 MB 00:00 2024-03-18T15:27:43.690 INFO:teuthology.orchestra.run.smithi137.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 950 kB/s | 103 kB 00:00 2024-03-18T15:27:43.723 INFO:teuthology.orchestra.run.smithi137.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-18T15:27:43.748 INFO:teuthology.orchestra.run.smithi137.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-03-18T15:27:43.820 INFO:teuthology.orchestra.run.smithi096.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 122 kB/s | 39 kB 00:00 2024-03-18T15:27:43.999 INFO:teuthology.orchestra.run.smithi137.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 681 kB/s | 187 kB 00:00 2024-03-18T15:27:44.037 INFO:teuthology.orchestra.run.smithi096.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-03-18T15:27:44.049 INFO:teuthology.orchestra.run.smithi137.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-03-18T15:27:44.061 INFO:teuthology.orchestra.run.smithi193.stdout:(104/174): ceph-common-19.0.0-2162.g3d78f5ba.el 1.3 MB/s | 21 MB 00:15 2024-03-18T15:27:44.062 INFO:teuthology.orchestra.run.smithi096.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 170 kB/s | 41 kB 00:00 2024-03-18T15:27:44.078 INFO:teuthology.orchestra.run.smithi193.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-03-18T15:27:44.087 INFO:teuthology.orchestra.run.smithi096.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-18T15:27:44.099 INFO:teuthology.orchestra.run.smithi137.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-03-18T15:27:44.114 INFO:teuthology.orchestra.run.smithi096.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-18T15:27:44.125 INFO:teuthology.orchestra.run.smithi137.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-18T15:27:44.138 INFO:teuthology.orchestra.run.smithi096.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-18T15:27:44.163 INFO:teuthology.orchestra.run.smithi096.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-03-18T15:27:44.189 INFO:teuthology.orchestra.run.smithi096.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-18T15:27:44.214 INFO:teuthology.orchestra.run.smithi096.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-18T15:27:44.217 INFO:teuthology.orchestra.run.smithi137.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 184 kB/s | 21 kB 00:00 2024-03-18T15:27:44.242 INFO:teuthology.orchestra.run.smithi137.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-03-18T15:27:44.272 INFO:teuthology.orchestra.run.smithi096.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-18T15:27:44.298 INFO:teuthology.orchestra.run.smithi096.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-18T15:27:44.323 INFO:teuthology.orchestra.run.smithi096.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-18T15:27:44.342 INFO:teuthology.orchestra.run.smithi137.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-18T15:27:44.348 INFO:teuthology.orchestra.run.smithi096.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-18T15:27:44.392 INFO:teuthology.orchestra.run.smithi137.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-03-18T15:27:44.451 INFO:teuthology.orchestra.run.smithi137.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 3.6 MB/s | 216 kB 00:00 2024-03-18T15:27:44.549 INFO:teuthology.orchestra.run.smithi096.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 833 kB/s | 187 kB 00:00 2024-03-18T15:27:44.586 INFO:teuthology.orchestra.run.smithi137.stdout:(100/174): openssl-devel-3.0.7-27.el9.x86_64.rp 11 MB/s | 4.1 MB 00:00 2024-03-18T15:27:44.615 INFO:teuthology.orchestra.run.smithi096.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-03-18T15:27:44.626 INFO:teuthology.orchestra.run.smithi137.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 199 kB/s | 35 kB 00:00 2024-03-18T15:27:44.641 INFO:teuthology.orchestra.run.smithi096.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-03-18T15:27:44.652 INFO:teuthology.orchestra.run.smithi137.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 666 kB/s | 44 kB 00:00 2024-03-18T15:27:44.666 INFO:teuthology.orchestra.run.smithi096.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-18T15:27:44.677 INFO:teuthology.orchestra.run.smithi137.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-18T15:27:44.744 INFO:teuthology.orchestra.run.smithi137.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-03-18T15:27:44.841 INFO:teuthology.orchestra.run.smithi096.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 107 kB/s | 21 kB 00:00 2024-03-18T15:27:44.866 INFO:teuthology.orchestra.run.smithi096.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 127 kB/s | 25 kB 00:00 2024-03-18T15:27:44.886 INFO:teuthology.orchestra.run.smithi137.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-03-18T15:27:45.000 INFO:teuthology.orchestra.run.smithi096.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-03-18T15:27:45.028 INFO:teuthology.orchestra.run.smithi193.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 325 kB/s | 309 kB 00:00 2024-03-18T15:27:45.117 INFO:teuthology.orchestra.run.smithi096.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 15 MB/s | 4.1 MB 00:00 2024-03-18T15:27:45.129 INFO:teuthology.orchestra.run.smithi193.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-03-18T15:27:45.142 INFO:teuthology.orchestra.run.smithi096.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-18T15:27:45.187 INFO:teuthology.orchestra.run.smithi193.stdout:(108/174): protobuf-compiler-3.14.0-13.el9.x86_ 468 kB/s | 863 kB 00:01 2024-03-18T15:27:45.194 INFO:teuthology.orchestra.run.smithi096.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.8 MB/s | 216 kB 00:00 2024-03-18T15:27:45.218 INFO:teuthology.orchestra.run.smithi096.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 463 kB/s | 35 kB 00:00 2024-03-18T15:27:45.221 INFO:teuthology.orchestra.run.smithi193.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 473 kB/s | 548 kB 00:01 2024-03-18T15:27:45.260 INFO:teuthology.orchestra.run.smithi096.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 665 kB/s | 44 kB 00:00 2024-03-18T15:27:45.285 INFO:teuthology.orchestra.run.smithi096.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-03-18T15:27:45.321 INFO:teuthology.orchestra.run.smithi193.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 487 kB/s | 49 kB 00:00 2024-03-18T15:27:45.410 INFO:teuthology.orchestra.run.smithi096.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-03-18T15:27:45.421 INFO:teuthology.orchestra.run.smithi193.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 673 kB/s | 67 kB 00:00 2024-03-18T15:27:45.627 INFO:teuthology.orchestra.run.smithi096.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-18T15:27:45.638 INFO:teuthology.orchestra.run.smithi193.stdout:(112/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 58 kB/s | 26 kB 00:00 2024-03-18T15:27:45.680 INFO:teuthology.orchestra.run.smithi193.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 584 kB/s | 151 kB 00:00 2024-03-18T15:27:45.745 INFO:teuthology.orchestra.run.smithi137.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 360 kB/s | 309 kB 00:00 2024-03-18T15:27:45.761 INFO:teuthology.orchestra.run.smithi096.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-18T15:27:45.811 INFO:teuthology.orchestra.run.smithi096.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-03-18T15:27:45.920 INFO:teuthology.orchestra.run.smithi096.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-18T15:27:45.978 INFO:teuthology.orchestra.run.smithi096.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-03-18T15:27:46.037 INFO:teuthology.orchestra.run.smithi096.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-03-18T15:27:46.096 INFO:teuthology.orchestra.run.smithi096.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-03-18T15:27:46.163 INFO:teuthology.orchestra.run.smithi096.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-18T15:27:46.178 INFO:teuthology.orchestra.run.smithi137.stdout:(107/174): ceph-test-19.0.0-2162.g3d78f5ba.el9. 4.4 MB/s | 48 MB 00:10 2024-03-18T15:27:46.204 INFO:teuthology.orchestra.run.smithi137.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-03-18T15:27:46.238 INFO:teuthology.orchestra.run.smithi137.stdout:(109/174): abseil-cpp-20211102.0-3.el9.x86_64.r 367 kB/s | 548 kB 00:01 2024-03-18T15:27:46.313 INFO:teuthology.orchestra.run.smithi137.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 238 kB/s | 26 kB 00:00 2024-03-18T15:27:46.338 INFO:teuthology.orchestra.run.smithi137.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-03-18T15:27:46.356 INFO:teuthology.orchestra.run.smithi193.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 811 kB/s | 548 kB 00:00 2024-03-18T15:27:46.371 INFO:teuthology.orchestra.run.smithi096.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-03-18T15:27:46.447 INFO:teuthology.orchestra.run.smithi193.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-03-18T15:27:46.497 INFO:teuthology.orchestra.run.smithi137.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-03-18T15:27:46.505 INFO:teuthology.orchestra.run.smithi096.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-03-18T15:27:46.538 INFO:teuthology.orchestra.run.smithi096.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-03-18T15:27:46.547 INFO:teuthology.orchestra.run.smithi193.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-03-18T15:27:46.564 INFO:teuthology.orchestra.run.smithi096.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-03-18T15:27:46.597 INFO:teuthology.orchestra.run.smithi096.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-18T15:27:46.606 INFO:teuthology.orchestra.run.smithi137.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-03-18T15:27:46.622 INFO:teuthology.orchestra.run.smithi096.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-03-18T15:27:46.656 INFO:teuthology.orchestra.run.smithi096.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-18T15:27:46.657 INFO:teuthology.orchestra.run.smithi193.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-03-18T15:27:46.723 INFO:teuthology.orchestra.run.smithi096.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-18T15:27:46.748 INFO:teuthology.orchestra.run.smithi096.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-03-18T15:27:46.784 INFO:teuthology.orchestra.run.smithi096.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 602 kB/s | 35 kB 00:00 2024-03-18T15:27:46.798 INFO:teuthology.orchestra.run.smithi193.stdout:(118/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 723 kB/s | 838 kB 00:01 2024-03-18T15:27:46.807 INFO:teuthology.orchestra.run.smithi096.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-18T15:27:46.866 INFO:teuthology.orchestra.run.smithi096.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-03-18T15:27:46.923 INFO:teuthology.orchestra.run.smithi193.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 644 kB/s | 172 kB 00:00 2024-03-18T15:27:46.924 INFO:teuthology.orchestra.run.smithi096.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-03-18T15:27:46.983 INFO:teuthology.orchestra.run.smithi096.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-03-18T15:27:47.025 INFO:teuthology.orchestra.run.smithi096.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-03-18T15:27:47.051 INFO:teuthology.orchestra.run.smithi096.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-03-18T15:27:47.084 INFO:teuthology.orchestra.run.smithi096.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-03-18T15:27:47.101 INFO:teuthology.orchestra.run.smithi096.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-03-18T15:27:47.160 INFO:teuthology.orchestra.run.smithi096.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 795 kB/s | 46 kB 00:00 2024-03-18T15:27:47.210 INFO:teuthology.orchestra.run.smithi096.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-03-18T15:27:47.235 INFO:teuthology.orchestra.run.smithi096.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-18T15:27:47.260 INFO:teuthology.orchestra.run.smithi096.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-03-18T15:27:47.311 INFO:teuthology.orchestra.run.smithi096.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-03-18T15:27:47.323 INFO:teuthology.orchestra.run.smithi193.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 682 kB/s | 358 kB 00:00 2024-03-18T15:27:47.336 INFO:teuthology.orchestra.run.smithi096.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-03-18T15:27:47.356 INFO:teuthology.orchestra.run.smithi137.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 730 kB/s | 548 kB 00:00 2024-03-18T15:27:47.361 INFO:teuthology.orchestra.run.smithi096.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-03-18T15:27:47.365 INFO:teuthology.orchestra.run.smithi193.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 686 kB/s | 303 kB 00:00 2024-03-18T15:27:47.395 INFO:teuthology.orchestra.run.smithi096.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-03-18T15:27:47.420 INFO:teuthology.orchestra.run.smithi096.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-03-18T15:27:47.432 INFO:teuthology.orchestra.run.smithi193.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-03-18T15:27:47.456 INFO:teuthology.orchestra.run.smithi137.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-03-18T15:27:47.462 INFO:teuthology.orchestra.run.smithi096.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-18T15:27:47.496 INFO:teuthology.orchestra.run.smithi096.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 781 kB/s | 59 kB 00:00 2024-03-18T15:27:47.521 INFO:teuthology.orchestra.run.smithi096.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 607 kB/s | 36 kB 00:00 2024-03-18T15:27:47.555 INFO:teuthology.orchestra.run.smithi096.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-03-18T15:27:47.564 INFO:teuthology.orchestra.run.smithi137.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-03-18T15:27:47.580 INFO:teuthology.orchestra.run.smithi096.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-18T15:27:47.614 INFO:teuthology.orchestra.run.smithi096.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-18T15:27:47.672 INFO:teuthology.orchestra.run.smithi096.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 343 kB/s | 20 kB 00:00 2024-03-18T15:27:47.673 INFO:teuthology.orchestra.run.smithi137.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 713 kB/s | 838 kB 00:01 2024-03-18T15:27:47.690 INFO:teuthology.orchestra.run.smithi137.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-03-18T15:27:47.698 INFO:teuthology.orchestra.run.smithi096.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-18T15:27:47.731 INFO:teuthology.orchestra.run.smithi096.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-03-18T15:27:47.741 INFO:teuthology.orchestra.run.smithi193.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 540 kB/s | 203 kB 00:00 2024-03-18T15:27:47.931 INFO:teuthology.orchestra.run.smithi096.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-18T15:27:47.949 INFO:teuthology.orchestra.run.smithi137.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 624 kB/s | 172 kB 00:00 2024-03-18T15:27:48.008 INFO:teuthology.orchestra.run.smithi193.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-03-18T15:27:48.117 INFO:teuthology.orchestra.run.smithi193.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-03-18T15:27:48.215 INFO:teuthology.orchestra.run.smithi137.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 682 kB/s | 358 kB 00:00 2024-03-18T15:27:48.225 INFO:teuthology.orchestra.run.smithi193.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-18T15:27:48.316 INFO:teuthology.orchestra.run.smithi137.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-03-18T15:27:48.334 INFO:teuthology.orchestra.run.smithi193.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-03-18T15:27:48.416 INFO:teuthology.orchestra.run.smithi137.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 649 kB/s | 303 kB 00:00 2024-03-18T15:27:48.431 INFO:teuthology.orchestra.run.smithi096.stdout:(150/174): librados2-19.0.0-2162.g3d78f5ba.el9. 4.8 MB/s | 3.4 MB 00:00 2024-03-18T15:27:48.442 INFO:teuthology.orchestra.run.smithi193.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-03-18T15:27:48.551 INFO:teuthology.orchestra.run.smithi193.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-03-18T15:27:48.590 INFO:teuthology.orchestra.run.smithi096.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-18T15:27:48.608 INFO:teuthology.orchestra.run.smithi137.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 694 kB/s | 203 kB 00:00 2024-03-18T15:27:48.649 INFO:teuthology.orchestra.run.smithi096.stdout:(152/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-03-18T15:27:48.899 INFO:teuthology.orchestra.run.smithi096.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 7.4 MB/s | 2.3 MB 00:00 2024-03-18T15:27:48.900 INFO:teuthology.orchestra.run.smithi137.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 496 kB/s | 144 kB 00:00 2024-03-18T15:27:48.941 INFO:teuthology.orchestra.run.smithi096.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 6.8 MB/s | 1.8 MB 00:00 2024-03-18T15:27:48.983 INFO:teuthology.orchestra.run.smithi096.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 513 kB/s | 42 kB 00:00 2024-03-18T15:27:49.000 INFO:teuthology.orchestra.run.smithi137.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-03-18T15:27:49.008 INFO:teuthology.orchestra.run.smithi096.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-03-18T15:27:49.033 INFO:teuthology.orchestra.run.smithi096.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-18T15:27:49.059 INFO:teuthology.orchestra.run.smithi096.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 725 kB/s | 36 kB 00:00 2024-03-18T15:27:49.084 INFO:teuthology.orchestra.run.smithi096.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-18T15:27:49.100 INFO:teuthology.orchestra.run.smithi137.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-18T15:27:49.109 INFO:teuthology.orchestra.run.smithi096.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-18T15:27:49.135 INFO:teuthology.orchestra.run.smithi096.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 460 kB/s | 23 kB 00:00 2024-03-18T15:27:49.160 INFO:teuthology.orchestra.run.smithi096.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-03-18T15:27:49.185 INFO:teuthology.orchestra.run.smithi096.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-03-18T15:27:49.201 INFO:teuthology.orchestra.run.smithi137.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-03-18T15:27:49.210 INFO:teuthology.orchestra.run.smithi096.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-03-18T15:27:49.236 INFO:teuthology.orchestra.run.smithi096.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-18T15:27:49.261 INFO:teuthology.orchestra.run.smithi096.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-03-18T15:27:49.319 INFO:teuthology.orchestra.run.smithi096.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-03-18T15:27:49.334 INFO:teuthology.orchestra.run.smithi137.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-03-18T15:27:49.411 INFO:teuthology.orchestra.run.smithi096.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-18T15:27:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-03-18T15:27:49.495 INFO:teuthology.orchestra.run.smithi096.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-18T15:27:49.529 INFO:teuthology.orchestra.run.smithi096.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-03-18T15:27:49.546 INFO:teuthology.orchestra.run.smithi096.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 626 kB/s | 31 kB 00:00 2024-03-18T15:27:49.621 INFO:teuthology.orchestra.run.smithi096.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-18T15:27:49.996 INFO:teuthology.orchestra.run.smithi096.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 165 kB/s | 74 kB 00:00 2024-03-18T15:27:50.209 INFO:teuthology.orchestra.run.smithi193.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 630 kB/s | 1.0 MB 00:01 2024-03-18T15:27:50.360 INFO:teuthology.orchestra.run.smithi193.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 308 kB/s | 46 kB 00:00 2024-03-18T15:27:50.502 INFO:teuthology.orchestra.run.smithi193.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 557 kB/s | 79 kB 00:00 2024-03-18T15:27:50.544 INFO:teuthology.orchestra.run.smithi193.stdout:(133/174): python3-grpcio-1.46.7-10.el9.x86_64. 671 kB/s | 2.0 MB 00:03 2024-03-18T15:27:50.644 INFO:teuthology.orchestra.run.smithi193.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 407 kB/s | 58 kB 00:00 2024-03-18T15:27:50.786 INFO:teuthology.orchestra.run.smithi193.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-03-18T15:27:51.028 INFO:teuthology.orchestra.run.smithi193.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 372 kB/s | 90 kB 00:00 2024-03-18T15:27:51.061 INFO:teuthology.orchestra.run.smithi193.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 526 kB/s | 272 kB 00:00 2024-03-18T15:27:51.145 INFO:teuthology.orchestra.run.smithi193.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-03-18T15:27:51.179 INFO:teuthology.orchestra.run.smithi193.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-03-18T15:27:51.304 INFO:teuthology.orchestra.run.smithi193.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-03-18T15:27:51.404 INFO:teuthology.orchestra.run.smithi193.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-03-18T15:27:51.454 INFO:teuthology.orchestra.run.smithi193.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 610 kB/s | 188 kB 00:00 2024-03-18T15:27:51.521 INFO:teuthology.orchestra.run.smithi193.stdout:(143/174): libarrow-9.0.0-10.el9.x86_64.rpm 708 kB/s | 4.4 MB 00:06 2024-03-18T15:27:51.548 INFO:teuthology.orchestra.run.smithi193.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 358 kB/s | 51 kB 00:00 2024-03-18T15:27:51.635 INFO:teuthology.orchestra.run.smithi137.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 483 kB/s | 1.0 MB 00:02 2024-03-18T15:27:51.722 INFO:teuthology.orchestra.run.smithi193.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 448 kB/s | 90 kB 00:00 2024-03-18T15:27:51.777 INFO:teuthology.orchestra.run.smithi137.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 621 kB/s | 2.0 MB 00:03 2024-03-18T15:27:51.794 INFO:teuthology.orchestra.run.smithi137.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 292 kB/s | 46 kB 00:00 2024-03-18T15:27:51.839 INFO:teuthology.orchestra.run.smithi193.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 598 kB/s | 230 kB 00:00 2024-03-18T15:27:51.865 INFO:teuthology.orchestra.run.smithi193.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-03-18T15:27:51.877 INFO:teuthology.orchestra.run.smithi137.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 788 kB/s | 79 kB 00:00 2024-03-18T15:27:51.894 INFO:teuthology.orchestra.run.smithi137.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-03-18T15:27:51.994 INFO:teuthology.orchestra.run.smithi137.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-03-18T15:27:52.106 INFO:teuthology.orchestra.run.smithi193.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 718 kB/s | 192 kB 00:00 2024-03-18T15:27:52.129 INFO:teuthology.orchestra.run.smithi096.stdout:(174/174): ceph-test-19.0.0-2162.g3d78f5ba.el9. 3.2 MB/s | 48 MB 00:14 2024-03-18T15:27:52.140 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-18T15:27:52.140 INFO:teuthology.orchestra.run.smithi096.stdout:Total 11 MB/s | 224 MB 00:20 2024-03-18T15:27:52.195 INFO:teuthology.orchestra.run.smithi137.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 449 kB/s | 90 kB 00:00 2024-03-18T15:27:52.248 INFO:teuthology.orchestra.run.smithi193.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 609 kB/s | 427 kB 00:00 2024-03-18T15:27:52.320 INFO:teuthology.orchestra.run.smithi137.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 226 kB/s | 28 kB 00:00 2024-03-18T15:27:52.345 INFO:teuthology.orchestra.run.smithi137.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 582 kB/s | 272 kB 00:00 2024-03-18T15:27:52.432 INFO:teuthology.orchestra.run.smithi193.stdout:(150/174): librados2-19.0.0-2162.g3d78f5ba.el9. 10 MB/s | 3.4 MB 00:00 2024-03-18T15:27:52.446 INFO:teuthology.orchestra.run.smithi137.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-03-18T15:27:52.498 INFO:teuthology.orchestra.run.smithi193.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-03-18T15:27:52.564 INFO:teuthology.orchestra.run.smithi137.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2024-03-18T15:27:52.594 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-03-18T15:27:52.688 INFO:teuthology.orchestra.run.smithi137.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 551 kB/s | 188 kB 00:00 2024-03-18T15:27:52.713 INFO:teuthology.orchestra.run.smithi137.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 238 kB/s | 36 kB 00:00 2024-03-18T15:27:52.805 INFO:teuthology.orchestra.run.smithi137.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-03-18T15:27:52.905 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-03-18T15:27:52.905 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-03-18T15:27:53.030 INFO:teuthology.orchestra.run.smithi137.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 399 kB/s | 90 kB 00:00 2024-03-18T15:27:53.089 INFO:teuthology.orchestra.run.smithi137.stdout:(145/174): libarrow-9.0.0-10.el9.x86_64.rpm 655 kB/s | 4.4 MB 00:06 2024-03-18T15:27:53.164 INFO:teuthology.orchestra.run.smithi137.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 509 kB/s | 230 kB 00:00 2024-03-18T15:27:53.215 INFO:teuthology.orchestra.run.smithi137.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-03-18T15:27:53.407 INFO:teuthology.orchestra.run.smithi193.stdout:(152/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 2.7 MB/s | 3.1 MB 00:01 2024-03-18T15:27:53.456 INFO:teuthology.orchestra.run.smithi137.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 657 kB/s | 192 kB 00:00 2024-03-18T15:27:53.740 INFO:teuthology.orchestra.run.smithi137.stdout:(149/174): python3-werkzeug-2.0.3-3.el9.1.noarc 602 kB/s | 427 kB 00:00 2024-03-18T15:27:54.140 INFO:teuthology.orchestra.run.smithi137.stdout:(150/174): librbd1-19.0.0-2162.g3d78f5ba.el9.x8 7.9 MB/s | 3.1 MB 00:00 2024-03-18T15:27:54.249 INFO:teuthology.orchestra.run.smithi137.stdout:(151/174): librados2-19.0.0-2162.g3d78f5ba.el9. 4.3 MB/s | 3.4 MB 00:00 2024-03-18T15:27:54.315 INFO:teuthology.orchestra.run.smithi193.stdout:(153/174): thrift-0.15.0-2.el9.x86_64.rpm 661 kB/s | 1.6 MB 00:02 2024-03-18T15:27:54.399 INFO:teuthology.orchestra.run.smithi137.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 1.2 MB/s | 321 kB 00:00 2024-03-18T15:27:54.912 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-03-18T15:27:54.912 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-03-18T15:27:54.915 INFO:teuthology.orchestra.run.smithi137.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 3.4 MB/s | 2.3 MB 00:00 2024-03-18T15:27:54.958 INFO:teuthology.orchestra.run.smithi137.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-03-18T15:27:55.000 INFO:teuthology.orchestra.run.smithi137.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 500 kB/s | 42 kB 00:00 2024-03-18T15:27:55.025 INFO:teuthology.orchestra.run.smithi137.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 3.6 MB/s | 248 kB 00:00 2024-03-18T15:27:55.041 INFO:teuthology.orchestra.run.smithi193.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 58 kB/s | 42 kB 00:00 2024-03-18T15:27:55.067 INFO:teuthology.orchestra.run.smithi137.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 71 kB 00:00 2024-03-18T15:27:55.092 INFO:teuthology.orchestra.run.smithi137.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 546 kB/s | 36 kB 00:00 2024-03-18T15:27:55.142 INFO:teuthology.orchestra.run.smithi137.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-18T15:27:55.168 INFO:teuthology.orchestra.run.smithi137.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 256 kB 00:00 2024-03-18T15:27:55.201 INFO:teuthology.orchestra.run.smithi137.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 394 kB/s | 23 kB 00:00 2024-03-18T15:27:55.226 INFO:teuthology.orchestra.run.smithi137.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 108 kB 00:00 2024-03-18T15:27:55.277 INFO:teuthology.orchestra.run.smithi137.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-18T15:27:55.294 INFO:teuthology.orchestra.run.smithi137.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 136 kB 00:00 2024-03-18T15:27:55.327 INFO:teuthology.orchestra.run.smithi137.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-18T15:27:55.369 INFO:teuthology.orchestra.run.smithi137.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 380 kB/s | 29 kB 00:00 2024-03-18T15:27:55.453 INFO:teuthology.orchestra.run.smithi137.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 9.6 MB/s | 1.2 MB 00:00 2024-03-18T15:27:55.512 INFO:teuthology.orchestra.run.smithi137.stdout:(168/174): thrift-0.15.0-2.el9.x86_64.rpm 705 kB/s | 1.6 MB 00:02 2024-03-18T15:27:55.670 INFO:teuthology.orchestra.run.smithi137.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.2 MB/s | 2.2 MB 00:00 2024-03-18T15:27:55.779 INFO:teuthology.orchestra.run.smithi137.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.0 MB/s | 2.3 MB 00:00 2024-03-18T15:27:55.796 INFO:teuthology.orchestra.run.smithi137.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 251 kB/s | 31 kB 00:00 2024-03-18T15:27:55.913 INFO:teuthology.orchestra.run.smithi137.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 1.1 MB/s | 465 kB 00:00 2024-03-18T15:27:55.955 INFO:teuthology.orchestra.run.smithi137.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-03-18T15:27:55.980 INFO:teuthology.orchestra.run.smithi137.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 405 kB/s | 74 kB 00:00 2024-03-18T15:27:55.987 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-03-18T15:27:55.988 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.4 MB/s | 224 MB 00:26 2024-03-18T15:27:56.024 INFO:teuthology.orchestra.run.smithi193.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 252 kB/s | 248 kB 00:00 2024-03-18T15:27:56.445 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-03-18T15:27:56.591 INFO:teuthology.orchestra.run.smithi193.stdout:(156/174): NetworkManager-1.46.0-1.el9.x86_64.r 573 kB/s | 2.3 MB 00:04 2024-03-18T15:27:56.633 INFO:teuthology.orchestra.run.smithi193.stdout:(157/174): NetworkManager-libnm-1.46.0-1.el9.x8 577 kB/s | 1.8 MB 00:03 2024-03-18T15:27:56.700 INFO:teuthology.orchestra.run.smithi193.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 106 kB/s | 71 kB 00:00 2024-03-18T15:27:56.714 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-03-18T15:27:56.714 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-03-18T15:27:57.201 INFO:teuthology.orchestra.run.smithi193.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 60 kB/s | 36 kB 00:00 2024-03-18T15:27:57.492 INFO:teuthology.orchestra.run.smithi193.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 224 kB/s | 192 kB 00:00 2024-03-18T15:27:57.651 INFO:teuthology.orchestra.run.smithi193.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 269 kB/s | 256 kB 00:00 2024-03-18T15:27:57.784 INFO:teuthology.orchestra.run.smithi193.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 40 kB/s | 23 kB 00:00 2024-03-18T15:27:58.276 INFO:teuthology.orchestra.run.smithi193.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 138 kB/s | 108 kB 00:00 2024-03-18T15:27:58.422 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-03-18T15:27:58.423 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-03-18T15:27:58.593 INFO:teuthology.orchestra.run.smithi193.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 164 kB/s | 155 kB 00:00 2024-03-18T15:27:58.626 INFO:teuthology.orchestra.run.smithi193.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 161 kB/s | 136 kB 00:00 2024-03-18T15:27:59.027 INFO:teuthology.orchestra.run.smithi193.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 85 kB/s | 64 kB 00:00 2024-03-18T15:27:59.244 INFO:teuthology.orchestra.run.smithi193.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 44 kB/s | 29 kB 00:00 2024-03-18T15:28:00.861 INFO:teuthology.orchestra.run.smithi193.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 555 kB/s | 1.2 MB 00:02 2024-03-18T15:28:01.887 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-03-18T15:28:02.000 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T15:28:02.026 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T15:28:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:02.103 INFO:teuthology.orchestra.run.smithi193.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 375 kB/s | 465 kB 00:01 2024-03-18T15:28:02.158 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:02.364 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T15:28:02.446 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T15:28:02.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T15:28:02.510 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T15:28:02.528 INFO:teuthology.orchestra.run.smithi193.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 631 kB/s | 2.2 MB 00:03 2024-03-18T15:28:02.566 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T15:28:02.728 INFO:teuthology.orchestra.run.smithi193.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 50 kB/s | 31 kB 00:00 2024-03-18T15:28:02.787 INFO:teuthology.orchestra.run.smithi193.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 660 kB/s | 2.3 MB 00:03 2024-03-18T15:28:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T15:28:02.945 INFO:teuthology.orchestra.run.smithi193.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 343 kB/s | 74 kB 00:00 2024-03-18T15:28:02.956 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:03.094 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:03.124 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:03.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:28:03.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T15:28:03.481 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T15:28:03.533 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T15:28:03.553 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:03.657 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:03.745 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:03.783 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T15:28:03.807 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:03.821 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:04.050 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:04.219 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T15:28:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T15:28:04.699 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:04.759 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:04.829 INFO:teuthology.orchestra.run.smithi193.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 478 kB/s | 1.1 MB 00:02 2024-03-18T15:28:04.835 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-03-18T15:28:04.835 INFO:teuthology.orchestra.run.smithi193.stdout:Total 6.1 MB/s | 224 MB 00:36 2024-03-18T15:28:04.875 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:04.927 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:05.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T15:28:05.308 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-03-18T15:28:05.352 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T15:28:05.383 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T15:28:05.409 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T15:28:05.444 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T15:28:05.500 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-03-18T15:28:05.518 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T15:28:05.579 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-03-18T15:28:05.579 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-03-18T15:28:05.580 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T15:28:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T15:28:05.666 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T15:28:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T15:28:05.701 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T15:28:05.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T15:28:05.740 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:05.749 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T15:28:05.798 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T15:28:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T15:28:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T15:28:05.851 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T15:28:05.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T15:28:05.911 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T15:28:05.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T15:28:05.952 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T15:28:05.953 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-18T15:28:06.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-18T15:28:06.025 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-18T15:28:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T15:28:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T15:28:06.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T15:28:06.273 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T15:28:06.332 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:06.346 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T15:28:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T15:28:06.441 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:06.471 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:06.545 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:06.575 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:28:06.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T15:28:06.781 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T15:28:06.830 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T15:28:06.851 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:06.919 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T15:28:06.963 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:06.985 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:06.995 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T15:28:07.049 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:07.079 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T15:28:07.089 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T15:28:07.112 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:07.123 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:07.127 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T15:28:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T15:28:07.243 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-18T15:28:07.269 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:07.273 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T15:28:07.289 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-03-18T15:28:07.289 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-03-18T15:28:07.300 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:07.341 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T15:28:07.445 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T15:28:07.526 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T15:28:07.545 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:07.605 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:07.677 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T15:28:07.781 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:07.817 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T15:28:07.861 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T15:28:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:08.011 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T15:28:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T15:28:08.094 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T15:28:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T15:28:08.174 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T15:28:08.189 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T15:28:08.207 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T15:28:08.219 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T15:28:08.243 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T15:28:08.246 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T15:28:08.282 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T15:28:08.290 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T15:28:08.326 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T15:28:08.355 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T15:28:08.369 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T15:28:08.410 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T15:28:08.411 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T15:28:08.448 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T15:28:08.520 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T15:28:08.544 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T15:28:08.556 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T15:28:08.592 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T15:28:08.601 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T15:28:08.622 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T15:28:08.632 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T15:28:08.660 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T15:28:08.689 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T15:28:08.729 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T15:28:08.765 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-18T15:28:08.782 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T15:28:08.808 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-18T15:28:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T15:28:08.837 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-18T15:28:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T15:28:08.888 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T15:28:08.904 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T15:28:08.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T15:28:08.951 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T15:28:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T15:28:09.024 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T15:28:09.131 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T15:28:09.158 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T15:28:09.212 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T15:28:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T15:28:09.288 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T15:28:09.336 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T15:28:09.392 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T15:28:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T15:28:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T15:28:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T15:28:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T15:28:09.783 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T15:28:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T15:28:09.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T15:28:09.853 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T15:28:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T15:28:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T15:28:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T15:28:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T15:28:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T15:28:09.986 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T15:28:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T15:28:10.017 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T15:28:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-18T15:28:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T15:28:10.092 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T15:28:10.127 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T15:28:10.161 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T15:28:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T15:28:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T15:28:10.316 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T15:28:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T15:28:10.406 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T15:28:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T15:28:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T15:28:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T15:28:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T15:28:10.552 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T15:28:10.599 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T15:28:10.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T15:28:10.676 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T15:28:10.965 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T15:28:11.166 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T15:28:11.221 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T15:28:11.294 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T15:28:11.334 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T15:28:11.370 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T15:28:11.405 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T15:28:11.416 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T15:28:11.456 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T15:28:11.461 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T15:28:11.496 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T15:28:11.498 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T15:28:11.538 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T15:28:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T15:28:11.625 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T15:28:11.663 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T15:28:11.673 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T15:28:11.719 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T15:28:11.749 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T15:28:11.790 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T15:28:11.840 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T15:28:11.869 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T15:28:11.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T15:28:11.924 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T15:28:11.946 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T15:28:11.964 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T15:28:11.990 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T15:28:12.014 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T15:28:12.086 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T15:28:12.215 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T15:28:12.324 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T15:28:12.363 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T15:28:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T15:28:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T15:28:12.599 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T15:28:12.649 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T15:28:12.693 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T15:28:12.708 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T15:28:12.821 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T15:28:12.858 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T15:28:12.890 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T15:28:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T15:28:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T15:28:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T15:28:13.022 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T15:28:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T15:28:13.084 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T15:28:13.160 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T15:28:13.194 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T15:28:13.227 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T15:28:13.268 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T15:28:13.303 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T15:28:13.391 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T15:28:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T15:28:13.525 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T15:28:13.557 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T15:28:13.604 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T15:28:13.700 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T15:28:13.758 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T15:28:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T15:28:14.029 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T15:28:14.068 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T15:28:14.099 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T15:28:14.127 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T15:28:14.141 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-03-18T15:28:14.265 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-18T15:28:14.308 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-18T15:28:14.339 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:14.347 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-18T15:28:14.398 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-18T15:28:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T15:28:14.452 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-18T15:28:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:14.482 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T15:28:14.502 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-18T15:28:14.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T15:28:14.528 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-18T15:28:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:14.556 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T15:28:14.562 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-18T15:28:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-18T15:28:14.604 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T15:28:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T15:28:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T15:28:14.732 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T15:28:14.895 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-18T15:28:14.909 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T15:28:14.938 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T15:28:14.955 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:15.015 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T15:28:15.075 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:15.082 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T15:28:15.106 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:15.110 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-18T15:28:15.187 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:15.218 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:28:15.253 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-18T15:28:15.429 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-18T15:28:15.483 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-18T15:28:15.511 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:15.599 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:15.620 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:15.647 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T15:28:15.676 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 17/198 2024-03-18T15:28:15.716 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-18T15:28:15.739 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:15.754 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-18T15:28:15.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T15:28:15.930 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:15.961 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-18T15:28:16.002 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-18T15:28:16.105 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-18T15:28:16.187 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:16.231 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-18T15:28:16.303 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:16.366 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:16.566 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-18T15:28:16.680 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-18T15:28:16.775 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-18T15:28:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-18T15:28:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-18T15:28:16.866 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-18T15:28:16.941 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-18T15:28:16.966 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T15:28:16.994 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-18T15:28:17.038 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T15:28:17.041 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-18T15:28:17.069 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T15:28:17.100 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T15:28:17.106 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-18T15:28:17.127 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T15:28:17.140 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-18T15:28:17.184 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-18T15:28:17.217 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-18T15:28:17.253 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-18T15:28:17.323 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-18T15:28:17.358 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 39/198 2024-03-18T15:28:17.401 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 40/198 2024-03-18T15:28:17.430 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 41/198 2024-03-18T15:28:17.438 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T15:28:17.440 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:17.475 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:17.475 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:17.492 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-18T15:28:17.545 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-18T15:28:17.547 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:17.579 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-18T15:28:17.615 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T15:28:17.619 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-18T15:28:17.751 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-18T15:28:17.806 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-18T15:28:18.025 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:18.062 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:18.075 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T15:28:18.111 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:18.153 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:18.153 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T15:28:18.153 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T15:28:18.153 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:18.164 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:18.188 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:18.188 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-18T15:28:18.327 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-18T15:28:18.410 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-18T15:28:18.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-18T15:28:18.530 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:18.538 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-18T15:28:18.570 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-18T15:28:18.611 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-18T15:28:18.649 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 53/198 2024-03-18T15:28:18.678 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-18T15:28:18.904 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-18T15:28:19.071 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-18T15:28:19.140 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-18T15:28:19.184 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-18T15:28:19.320 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-18T15:28:19.387 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-18T15:28:19.442 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-18T15:28:19.521 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-18T15:28:19.560 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-18T15:28:19.597 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-18T15:28:19.644 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-18T15:28:19.680 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-18T15:28:19.722 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-18T15:28:19.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-18T15:28:19.898 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-18T15:28:19.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-18T15:28:19.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-18T15:28:20.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-18T15:28:20.119 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-18T15:28:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-18T15:28:20.191 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-18T15:28:20.217 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-18T15:28:20.241 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-18T15:28:20.312 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-18T15:28:20.432 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-18T15:28:20.450 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:20.484 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:20.540 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-18T15:28:20.583 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-18T15:28:20.631 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-18T15:28:20.686 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-18T15:28:20.870 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-18T15:28:20.921 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-18T15:28:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:21.049 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-18T15:28:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:21.086 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-18T15:28:21.117 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-18T15:28:21.142 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:21.143 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T15:28:21.143 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T15:28:21.143 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:21.149 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-18T15:28:21.163 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:21.177 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-18T15:28:21.211 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-18T15:28:21.250 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-18T15:28:21.282 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-18T15:28:21.313 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-18T15:28:21.388 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-18T15:28:21.422 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-18T15:28:21.456 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-18T15:28:21.497 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-18T15:28:21.532 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-18T15:28:21.622 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-18T15:28:21.712 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-18T15:28:21.755 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-18T15:28:21.786 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-18T15:28:21.834 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-18T15:28:21.931 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-18T15:28:21.988 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-18T15:28:22.731 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-18T15:28:22.777 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-18T15:28:22.811 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-18T15:28:22.860 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-18T15:28:22.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-18T15:28:22.987 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-18T15:28:23.165 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-18T15:28:23.193 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-18T15:28:23.270 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-18T15:28:23.912 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-18T15:28:24.016 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-18T15:28:25.227 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-18T15:28:25.311 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-18T15:28:25.342 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-18T15:28:25.374 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-18T15:28:25.401 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-18T15:28:25.718 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-18T15:28:25.757 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:25.872 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 124/198 2024-03-18T15:28:25.903 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 125/198 2024-03-18T15:28:25.939 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-18T15:28:26.406 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-18T15:28:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:26.505 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 128/198 2024-03-18T15:28:26.528 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 129/198 2024-03-18T15:28:28.754 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:28.800 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:29.342 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 130/198 2024-03-18T15:28:29.386 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:29.467 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 131/198 2024-03-18T15:28:29.467 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-18T15:28:29.467 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-18T15:28:29.468 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:29.505 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-03-18T15:28:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-03-18T15:28:29.606 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-03-18T15:28:29.606 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-03-18T15:28:29.606 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:29.783 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:29.784 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T15:28:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:31.272 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:31.303 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:31.303 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:31.303 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T15:28:31.303 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:31.303 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:31.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:31.534 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:31.560 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:31.560 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:31.560 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T15:28:31.560 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:31.560 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:31.561 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:31.999 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:32.022 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:32.022 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:32.022 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T15:28:32.022 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:32.022 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:32.023 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:32.109 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T15:28:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:32.616 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-18T15:28:32.676 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-18T15:28:32.706 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:32.938 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:32.954 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:32.954 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-03-18T15:28:32.954 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-03-18T15:28:32.954 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-03-18T15:28:32.955 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-03-18T15:28:32.955 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-03-18T15:28:32.955 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-03-18T15:28:32.955 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-03-18T15:28:32.955 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:32.974 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:33.117 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:33.131 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:33.132 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:33.132 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T15:28:33.132 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:33.933 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:33.966 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:34.066 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-18T15:28:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:34.282 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:34.282 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:34.282 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T15:28:34.282 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:34.283 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:34.283 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:34.327 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:34.338 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:34.388 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-18T15:28:34.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:34.636 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:34.636 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:34.636 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T15:28:34.636 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:34.636 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:34.637 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:34.862 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:34.884 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:34.884 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:34.884 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T15:28:34.885 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:34.885 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:34.885 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:35.252 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T15:28:35.279 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:35.301 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:35.301 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:35.301 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T15:28:35.302 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:35.302 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:35.302 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:35.338 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:35.361 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:35.368 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:35.368 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T15:28:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:35.436 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:35.449 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T15:28:35.740 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:35.762 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:35.841 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:35.872 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:35.904 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-18T15:28:35.964 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-18T15:28:35.987 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:36.219 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:36.254 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:37.212 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:37.246 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:37.345 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:37.509 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-18T15:28:37.540 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:37.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:37.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:37.618 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:37.664 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-18T15:28:38.530 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T15:28:38.559 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:38.582 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:38.627 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:38.649 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:38.649 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:38.649 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T15:28:38.649 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:38.711 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:38.733 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:39.045 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:39.067 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:39.067 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T15:28:39.067 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:39.067 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:40.210 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:28:40.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-18T15:28:40.291 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-18T15:28:40.317 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T15:28:40.498 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-18T15:28:40.621 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T15:28:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-18T15:28:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T15:28:40.770 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T15:28:40.803 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T15:28:40.837 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-18T15:28:40.876 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T15:28:40.906 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-18T15:28:40.980 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-18T15:28:41.051 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T15:28:41.067 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 132/198 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-03-18T15:28:41.068 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:41.088 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T15:28:41.237 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:41.255 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 133/198 2024-03-18T15:28:41.255 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:41.255 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-18T15:28:41.255 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:41.713 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:41.773 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:42.590 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-18T15:28:42.665 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T15:28:42.706 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T15:28:42.741 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 134/198 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-18T15:28:42.766 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:42.796 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T15:28:42.828 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T15:28:42.852 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T15:28:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T15:28:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T15:28:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:42.999 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:43.009 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 135/198 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-18T15:28:43.022 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T15:28:43.039 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:43.123 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:43.158 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T15:28:43.191 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T15:28:43.217 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T15:28:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T15:28:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T15:28:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T15:28:43.364 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T15:28:43.389 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T15:28:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T15:28:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:28:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T15:28:43.467 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:43.482 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 136/198 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-18T15:28:43.489 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:43.524 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-18T15:28:43.554 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-18T15:28:43.577 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-18T15:28:43.581 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T15:28:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:43.725 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:28:43.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-18T15:28:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T15:28:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T15:28:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T15:28:43.881 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T15:28:43.901 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:28:43.929 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-18T15:28:43.968 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T15:28:43.970 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:44.001 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 138/198 2024-03-18T15:28:44.022 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T15:28:44.033 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 139/198 2024-03-18T15:28:44.055 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T15:28:44.079 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-18T15:28:44.092 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 140/198 2024-03-18T15:28:44.112 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T15:28:44.115 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:44.141 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-18T15:28:44.206 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-18T15:28:44.264 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T15:28:44.298 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T15:28:44.350 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 141/198 2024-03-18T15:28:44.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:44.909 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:44.946 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:44.966 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:45.356 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 142/198 2024-03-18T15:28:45.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:45.483 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 143/198 2024-03-18T15:28:45.648 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 144/198 2024-03-18T15:28:45.669 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:45.692 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 145/198 2024-03-18T15:28:45.692 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:45.692 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-18T15:28:45.692 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:45.692 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-18T15:28:45.693 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:45.747 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:45.758 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 146/198 2024-03-18T15:28:45.782 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-18T15:28:45.802 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 147/198 2024-03-18T15:28:45.877 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T15:28:45.918 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T15:28:45.998 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T15:28:46.030 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T15:28:46.055 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T15:28:46.081 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T15:28:46.107 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T15:28:46.107 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:46.211 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T15:28:46.241 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:46.334 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:46.361 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T15:28:46.394 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T15:28:46.419 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T15:28:46.444 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T15:28:46.477 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T15:28:46.516 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T15:28:46.549 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T15:28:46.574 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T15:28:46.599 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T15:28:46.625 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:28:46.650 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T15:28:46.656 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:46.656 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:46.689 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-18T15:28:46.724 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:46.746 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 149/198 2024-03-18T15:28:46.746 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:46.747 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-18T15:28:46.747 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:46.747 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:28:46.747 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:46.791 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:46.814 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 150/198 2024-03-18T15:28:46.814 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:46.814 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-18T15:28:46.814 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:46.836 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:46.836 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:46.858 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:28:46.881 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:46.911 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:46.935 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:28:47.005 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T15:28:47.044 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T15:28:47.070 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T15:28:47.134 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:28:47.201 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:47.223 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 152/198 2024-03-18T15:28:47.224 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-18T15:28:47.224 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-18T15:28:47.224 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:47.224 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-18T15:28:47.224 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-18T15:28:48.827 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-18T15:28:48.828 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-18T15:28:48.829 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-18T15:28:48.845 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T15:28:48.846 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T15:28:48.847 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T15:28:48.848 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T15:28:48.849 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T15:28:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T15:28:48.851 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T15:28:48.852 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T15:28:48.853 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:48.854 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T15:28:48.855 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T15:28:48.856 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T15:28:48.857 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T15:28:48.858 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T15:28:48.859 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T15:28:51.722 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:28:51.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 154/198 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T15:28:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T15:28:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T15:28:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: jq-1.6-16.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T15:28:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T15:28:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T15:28:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T15:28:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T15:28:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T15:28:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T15:28:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T15:28:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T15:28:51.809 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 155/198 2024-03-18T15:28:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T15:28:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:28:51.809 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-03-18T15:28:51.821 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-18T15:28:51.895 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-18T15:28:51.896 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-18T15:28:51.897 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T15:28:51.913 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T15:28:51.914 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T15:28:51.915 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T15:28:51.916 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T15:28:51.917 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T15:28:51.918 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T15:28:51.919 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T15:28:51.920 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-18T15:28:51.921 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T15:28:51.922 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T15:28:51.923 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T15:28:51.924 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T15:28:52.003 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 157/198 2024-03-18T15:28:52.123 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-18T15:28:52.161 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 159/198 2024-03-18T15:28:52.200 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-18T15:28:52.270 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-18T15:28:52.280 DEBUG:teuthology.parallel:result is None 2024-03-18T15:28:52.305 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-18T15:28:52.329 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 163/198 2024-03-18T15:28:52.386 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-18T15:28:52.416 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-18T15:28:52.470 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-18T15:28:52.531 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-18T15:28:52.565 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-18T15:28:53.184 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:53.222 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:53.241 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-18T15:28:54.045 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 170/198 2024-03-18T15:28:54.134 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-18T15:28:54.175 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-18T15:28:54.255 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-18T15:28:54.305 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-18T15:28:54.338 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-18T15:28:54.363 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-18T15:28:54.390 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-18T15:28:54.391 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:54.478 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-18T15:28:54.508 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-18T15:28:54.508 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:54.592 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-18T15:28:54.618 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-18T15:28:54.652 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-18T15:28:54.678 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-18T15:28:54.702 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-18T15:28:54.735 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-18T15:28:54.775 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-18T15:28:54.808 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-18T15:28:54.833 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-18T15:28:54.858 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-18T15:28:54.883 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:28:54.909 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-18T15:28:54.915 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:54.915 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:55.037 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T15:28:55.037 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:55.037 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-03-18T15:28:55.037 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:55.037 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.038 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T15:28:55.039 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.040 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.041 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-16.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.042 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T15:28:55.043 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T15:28:55.044 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T15:28:55.045 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T15:28:55.046 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T15:28:55.047 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T15:28:55.048 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T15:28:55.048 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T15:28:55.048 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T15:28:55.048 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T15:28:55.048 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T15:28:55.049 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.050 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T15:28:55.051 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T15:28:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T15:28:55.053 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-18T15:28:55.053 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-03-18T15:28:55.095 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-18T15:28:55.095 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:55.178 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-18T15:28:55.203 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:28:55.228 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-18T15:28:55.260 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-18T15:28:55.296 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-18T15:28:55.359 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:28:55.505 DEBUG:teuthology.parallel:result is None 2024-03-18T15:29:00.279 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 1/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 3/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 4/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2162.g3d78f 5/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 6/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 7/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 8/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 9/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 10/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 11/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 12/198 2024-03-18T15:29:00.280 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 13/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 14/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_ 15/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 16/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 17/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_6 18/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86 19/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 20/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el 21/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9. 22/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_6 23/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 24/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 25/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 26/198 2024-03-18T15:29:00.281 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 27/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 28/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 29/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba. 30/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noa 31/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.n 32/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d7 33/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el 34/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 35/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.e 36/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 37/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 38/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 39/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-18T15:29:00.282 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-18T15:29:00.283 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-18T15:29:00.284 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-18T15:29:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-18T15:29:00.286 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-18T15:29:00.287 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-18T15:29:00.288 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-18T15:29:00.289 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 151/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 153/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-18T15:29:00.290 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-18T15:29:00.291 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-18T15:29:00.292 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-18T15:29:00.293 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-18T15:29:03.232 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-18T15:29:03.232 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:29:03.232 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: acl-2.3.1-4.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-18T15:29:03.233 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.234 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.235 INFO:teuthology.orchestra.run.smithi193.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.236 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.237 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.238 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.239 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-2162.g3d78f5ba.el9.noarch 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-18T15:29:03.240 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: jq-1.6-16.el9.x86_64 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.241 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.242 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-18T15:29:03.243 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-18T15:29:03.244 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.11-1.el9.x86_64 2024-03-18T15:29:03.245 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-18T15:29:03.245 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-18T15:29:03.245 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-18T15:29:03.245 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-18T15:29:03.245 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.246 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.247 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-18T15:29:03.247 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-18T15:29:03.247 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-18T15:29:03.247 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-18T15:29:03.247 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-18T15:29:03.248 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-18T15:29:03.249 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-18T15:29:03.250 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-18T15:29:03.251 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-18T15:29:03.252 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-18T15:29:03.253 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-2162.g3d78f5ba.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-18T15:29:03.254 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-18T15:29:03.255 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-18T15:29:03.255 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:29:03.255 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-03-18T15:29:03.724 DEBUG:teuthology.parallel:result is None 2024-03-18T15:29:03.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:04.039 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T15:29:04.066 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.066 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.066 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-18T15:29:04.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:04.336 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T15:29:04.361 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.362 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.362 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-18T15:29:04.364 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:04.510 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-18T15:29:04.536 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.537 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162.g3d78f5ba.el9 2024-03-18T15:29:04.537 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162.g3d78f5ba is installed. 2024-03-18T15:29:04.539 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T15:29:04.539 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:29:04.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T15:29:04.574 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:29:04.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T15:29:04.611 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:29:04.611 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T15:29:04.645 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T15:29:04.646 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:29:04.646 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T15:29:04.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T15:29:04.747 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:29:04.748 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T15:29:04.779 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T15:29:04.848 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:29:04.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T15:29:04.880 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T15:29:04.951 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T15:29:04.951 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:29:04.951 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T15:29:04.983 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T15:29:05.052 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:29:05.052 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T15:29:05.080 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T15:29:05.148 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:29:05.148 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T15:29:05.179 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T15:29:05.249 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T15:29:05.250 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:29:05.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T15:29:05.277 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T15:29:05.345 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:29:05.345 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T15:29:05.372 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T15:29:05.439 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:29:05.439 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T15:29:05.470 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T15:29:05.541 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-18T15:29:05.663 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'mons down, quorum', 'out of quorum', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'subvols': {'create': 2}} 2024-03-18T15:29:05.663 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:05.664 INFO:tasks.cephadm:Cluster fsid is 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:29:05.664 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-18T15:29:05.664 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.96', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.193'} 2024-03-18T15:29:05.664 INFO:tasks.cephadm:First mon is mon.a on smithi096 2024-03-18T15:29:05.665 INFO:tasks.cephadm:First mgr is x 2024-03-18T15:29:05.665 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-18T15:29:05.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-03-18T15:29:05.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-03-18T15:29:05.723 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-03-18T15:29:05.756 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-18T15:29:05.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:05.901 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77658/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2162-g3d78f5ba', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2162.g3d78f5ba'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-15 01:33:04.210510', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240314.183243', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-18T15:29:06.051 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20240314.183243, sha1 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:06.053 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm 2024-03-18T15:29:06.053 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:06.670 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 18 15:29 /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:06.671 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.123 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 18 15:29 /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.123 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.801 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 767536 Mar 18 15:29 /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.801 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.820 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.839 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-18T15:29:07.874 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af on all hosts... 2024-03-18T15:29:07.874 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-18T15:29:07.877 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-18T15:29:07.882 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af pull 2024-03-18T15:29:08.770 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-18T15:29:08.818 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-18T15:29:08.865 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-18T15:29:45.948 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-18T15:29:45.949 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-18T15:29:52.229 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:29:52.390 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-18T15:29:52.390 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-18T15:29:52.391 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae", 2024-03-18T15:29:52.391 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-03-18T15:29:52.391 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513" 2024-03-18T15:29:52.391 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-03-18T15:29:52.391 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-18T15:29:52.416 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-03-18T15:29:52.447 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-03-18T15:29:52.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-03-18T15:29:52.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-03-18T15:29:52.539 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-03-18T15:29:52.570 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-03-18T15:29:52.597 INFO:tasks.cephadm:Writing seed config... 2024-03-18T15:29:52.598 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-18T15:29:52.598 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-18T15:29:52.598 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-18T15:29:52.598 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-18T15:29:52.599 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-18T15:29:52.600 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-18T15:29:52.601 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-18T15:29:52.602 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:29:52.602 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-18T15:29:52.617 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 419c928c-e53c-11ee-95c9-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-18T15:29:52.617 DEBUG:teuthology.orchestra.run.smithi096:mon.a> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service 2024-03-18T15:29:52.660 DEBUG:teuthology.orchestra.run.smithi096:mgr.x> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x.service 2024-03-18T15:29:52.703 INFO:tasks.cephadm:Bootstrapping... 2024-03-18T15:29:52.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af -v bootstrap --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.96 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:29:52.894 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-03-18T15:29:52.894 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', '-v', 'bootstrap', '--fsid', '419c928c-e53c-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.96', '--skip-admin-label'] 2024-03-18T15:29:52.918 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T15:29:52.919 INFO:teuthology.orchestra.run.smithi096.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-18T15:29:52.919 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying podman|docker is present... 2024-03-18T15:29:52.942 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T15:29:52.942 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying lvm2 is present... 2024-03-18T15:29:52.942 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying time synchronization is in place... 2024-03-18T15:29:52.949 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T15:29:52.949 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T15:29:52.955 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T15:29:52.955 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:29:52.961 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-03-18T15:29:52.967 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-03-18T15:29:52.967 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-03-18T15:29:52.967 INFO:teuthology.orchestra.run.smithi096.stdout:Repeating the final host check... 2024-03-18T15:29:52.989 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-18T15:29:52.989 INFO:teuthology.orchestra.run.smithi096.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-18T15:29:52.989 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl is present 2024-03-18T15:29:52.989 INFO:teuthology.orchestra.run.smithi096.stdout:lvcreate is present 2024-03-18T15:29:52.995 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-18T15:29:52.996 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-18T15:29:53.001 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-18T15:29:53.001 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:29:53.007 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Host looks OK 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Cluster fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Acquiring lock 140666880933312 on /run/cephadm/419c928c-e53c-11ee-95c9-87774f69a715.lock 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Lock 140666880933312 acquired on /run/cephadm/419c928c-e53c-11ee-95c9-87774f69a715.lock 2024-03-18T15:29:53.014 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 3300 ... 2024-03-18T15:29:53.015 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 6789 ... 2024-03-18T15:29:53.015 INFO:teuthology.orchestra.run.smithi096.stdout:Base mon IP(s) is [172.21.15.96:3300, 172.21.15.96:6789], mon addrv is [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-03-18T15:29:53.018 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.96 metric 100 2024-03-18T15:29:53.018 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.96 metric 100 2024-03-18T15:29:53.020 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-18T15:29:53.020 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-18T15:29:53.020 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2024-03-18T15:29:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-18T15:29:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-18T15:29:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T15:29:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-18T15:29:53.028 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-03-18T15:29:53.029 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-18T15:29:53.029 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-03-18T15:29:53.029 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-03-18T15:29:53.029 INFO:teuthology.orchestra.run.smithi096.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-18T15:29:53.029 INFO:teuthology.orchestra.run.smithi096.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-18T15:29:53.030 INFO:teuthology.orchestra.run.smithi096.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-18T15:29:53.526 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af... 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Getting image source signatures 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:66487374cb611c95435850c17b53ebb64c22ed9555f72cd43d678b60262a12b1 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying config sha256:be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae 2024-03-18T15:29:53.527 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-18T15:29:54.430 INFO:teuthology.orchestra.run.smithi096.stdout:ceph: stdout ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-18T15:29:54.430 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph version: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev) 2024-03-18T15:29:54.431 INFO:teuthology.orchestra.run.smithi096.stdout:Extracting ceph user uid/gid from container image... 2024-03-18T15:29:55.411 INFO:teuthology.orchestra.run.smithi096.stdout:stat: stdout 167 167 2024-03-18T15:29:55.411 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial keys... 2024-03-18T15:29:56.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQDzXfhlgj/eKhAAvk8O8UAZZMfpB4tDIgDNJA== 2024-03-18T15:29:57.231 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQD0XfhlEwlTIhAA28khuyz2rd9+iqkWpPO/RA== 2024-03-18T15:29:58.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQD1XfhlK6DDIBAAO/zU/Dw7J9P2/vzP4eshiw== 2024-03-18T15:29:58.263 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial monmap... 2024-03-18T15:29:59.126 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool for a [v2:172.21.15.96:3300,v1:172.21.15.96:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: set fsid to 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:29:59.127 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mon... 2024-03-18T15:30:00.018 INFO:teuthology.orchestra.run.smithi096.stdout:create mon.a on 2024-03-18T15:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-18T15:30:00.391 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-18T15:30:00.615 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target. 2024-03-18T15:30:00.615 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target. 2024-03-18T15:30:00.852 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a 2024-03-18T15:30:00.852 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service: Unit ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service not loaded. 2024-03-18T15:30:01.020 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target.wants/ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715@.service. 2024-03-18T15:30:01.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:01 smithi096 podman[31534]: 2024-03-18 15:30:01.478616328 +0000 UTC m=+0.346367890 container start b9728c0c4693fbeaf66397ceddf1aef8a7af6482fa836ef6cfada93a52665012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-18T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T15:30:01.580 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-03-18T15:30:01.586 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T15:30:01.587 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:30:01.587 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-03-18T15:30:01.587 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T15:30:01.587 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon to start... 2024-03-18T15:30:01.587 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon... 2024-03-18T15:30:01.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:01 smithi096 ceph-mon[31553]: mkfs 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:01.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:01 smithi096 ceph-mon[31553]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T15:30:01.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:01 smithi096 bash[31534]: b9728c0c4693fbeaf66397ceddf1aef8a7af6482fa836ef6cfada93a52665012 2024-03-18T15:30:01.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:01 smithi096 systemd[1]: Started Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:30:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout cluster: 2024-03-18T15:30:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout id: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-18T15:30:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout services: 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.515931s) 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout data: 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pgs: 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.991+0000 7f468d3fa700 1 Processor -- start 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.991+0000 7f468d3fa700 1 -- start start 2024-03-18T15:30:02.650 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.991+0000 7f468d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.991+0000 7f468d3fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4688107940 con 0x7f46881093b0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f4686ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f4686ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36208/0 (socket says 172.21.15.96:36208) 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f4686ffd700 1 -- 172.21.15.96:0/473961971 learned_addr learned my addr 172.21.15.96:0/473961971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f4686ffd700 1 -- 172.21.15.96:0/473961971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4688107ac0 con 0x7f46881093b0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f4686ffd700 1 --2- 172.21.15.96:0/473961971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46780098a0 tx=0x7f4678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c79297a1b91ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f46867fc700 1 -- 172.21.15.96:0/473961971 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678004160 con 0x7f46881093b0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f46867fc700 1 -- 172.21.15.96:0/473961971 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4678004300 con 0x7f46881093b0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.992+0000 7f46867fc700 1 -- 172.21.15.96:0/473961971 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46780036a0 con 0x7f46881093b0 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 -- 172.21.15.96:0/473961971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 msgr2=0x7f46881097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 --2- 172.21.15.96:0/473961971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f46780098a0 tx=0x7f4678009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 -- 172.21.15.96:0/473961971 shutdown_connections 2024-03-18T15:30:02.651 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 --2- 172.21.15.96:0/473961971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f46881097d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 -- 172.21.15.96:0/473961971 >> 172.21.15.96:0/473961971 conn(0x7f4688103320 msgr2=0x7f4688105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 -- 172.21.15.96:0/473961971 shutdown_connections 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 -- 172.21.15.96:0/473961971 wait complete. 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.993+0000 7f468d3fa700 1 Processor -- start 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f468d3fa700 1 -- start start 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f468d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f468d3fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4688107ff0 con 0x7f46881093b0 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4686ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4686ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36212/0 (socket says 172.21.15.96:36212) 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4686ffd700 1 -- 172.21.15.96:0/2152041857 learned_addr learned my addr 172.21.15.96:0/2152041857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4686ffd700 1 -- 172.21.15.96:0/2152041857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468819b9a0 con 0x7f46881093b0 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4686ffd700 1 --2- 172.21.15.96:0/2152041857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f46881089b0 tx=0x7f4678004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678003db0 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4678011630 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468819bc30 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678011930 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.994+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468819e920 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.995+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4678011ad0 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.995+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f467801b450 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.995+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4688063900 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:01.998+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4678004260 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.067+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f468802dc10 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.067+0000 7f4684ff9700 1 -- 172.21.15.96:0/2152041857 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4678025480 con 0x7f46881093b0 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 msgr2=0x7f468819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 --2- 172.21.15.96:0/2152041857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f46881089b0 tx=0x7f4678004750 comp rx=0 tx=0).stop 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 shutdown_connections 2024-03-18T15:30:02.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 --2- 172.21.15.96:0/2152041857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46881093b0 0x7f468819b460 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:02.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 >> 172.21.15.96:0/2152041857 conn(0x7f4688103320 msgr2=0x7f4688192120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:02.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 shutdown_connections 2024-03-18T15:30:02.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:02.068+0000 7f468d3fa700 1 -- 172.21.15.96:0/2152041857 wait complete. 2024-03-18T15:30:02.654 INFO:teuthology.orchestra.run.smithi096.stdout:mon is available 2024-03-18T15:30:02.654 INFO:teuthology.orchestra.run.smithi096.stdout:Assimilating anything we can from ceph.conf... 2024-03-18T15:30:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T15:30:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: monmap epoch 1 2024-03-18T15:30:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: last_changed 2024-03-18T15:29:58.578642+0000 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: min_mon_release 19 (squid) 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: election_strategy: 1 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: fsmap 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T15:30:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: mgrmap e1: no daemons active 2024-03-18T15:30:02.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:02 smithi096 ceph-mon[31553]: from='client.? 172.21.15.96:0/2152041857' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-18T15:30:03.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T15:30:03.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.072+0000 7f717e4cf700 1 Processor -- start 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.072+0000 7f717e4cf700 1 -- start start 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f717e4cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f717e4cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7178107660 con 0x7f7178106c70 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7177fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7177fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36216/0 (socket says 172.21.15.96:36216) 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7177fff700 1 -- 172.21.15.96:0/336524153 learned_addr learned my addr 172.21.15.96:0/336524153 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7177fff700 1 -- 172.21.15.96:0/336524153 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7178107e70 con 0x7f7178106c70 2024-03-18T15:30:03.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7177fff700 1 --2- 172.21.15.96:0/336524153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71680098a0 tx=0x7f7168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62dea0a880cc4c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7176ffd700 1 -- 172.21.15.96:0/336524153 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7168004160 con 0x7f7178106c70 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7176ffd700 1 -- 172.21.15.96:0/336524153 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7168004300 con 0x7f7178106c70 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.073+0000 7f7176ffd700 1 -- 172.21.15.96:0/336524153 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71680036a0 con 0x7f7178106c70 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 -- 172.21.15.96:0/336524153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 msgr2=0x7f7178107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 --2- 172.21.15.96:0/336524153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f71680098a0 tx=0x7f7168009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 -- 172.21.15.96:0/336524153 shutdown_connections 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 --2- 172.21.15.96:0/336524153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f7178107090 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 -- 172.21.15.96:0/336524153 >> 172.21.15.96:0/336524153 conn(0x7f7178102250 msgr2=0x7f7178104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 -- 172.21.15.96:0/336524153 shutdown_connections 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.074+0000 7f717e4cf700 1 -- 172.21.15.96:0/336524153 wait complete. 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f717e4cf700 1 Processor -- start 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f717e4cf700 1 -- start start 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f717e4cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f717e4cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71781083a0 con 0x7f7178106c70 2024-03-18T15:30:03.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f7177fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f7177fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36228/0 (socket says 172.21.15.96:36228) 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f7177fff700 1 -- 172.21.15.96:0/496297364 learned_addr learned my addr 172.21.15.96:0/496297364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f7177fff700 1 -- 172.21.15.96:0/496297364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717819bce0 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.075+0000 7f7177fff700 1 --2- 172.21.15.96:0/496297364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f7168004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7168003db0 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7168011630 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717819bf70 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7168011950 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717819ec60 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7168011b80 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.076+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f716801c450 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.077+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7178063900 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.079+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7168032020 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.145+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f717819c450 con 0x7f7178106c70 2024-03-18T15:30:03.752 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.148+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f717819c450 con 0x7f7178106c70 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f71757fa700 1 -- 172.21.15.96:0/496297364 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7168004260 con 0x7f7178106c70 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 msgr2=0x7f717819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f717e4cf700 1 --2- 172.21.15.96:0/496297364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7168000c00 tx=0x7f7168004750 comp rx=0 tx=0).stop 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 shutdown_connections 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f717e4cf700 1 --2- 172.21.15.96:0/496297364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7178106c70 0x7f717819b7a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.149+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 >> 172.21.15.96:0/496297364 conn(0x7f7178102250 msgr2=0x7f717807afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.150+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 shutdown_connections 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:03.150+0000 7f717e4cf700 1 -- 172.21.15.96:0/496297364 wait complete. 2024-03-18T15:30:03.753 INFO:teuthology.orchestra.run.smithi096.stdout:Generating new minimal ceph.conf... 2024-03-18T15:30:03.856 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:03 smithi096 ceph-mon[31553]: from='client.? 172.21.15.96:0/496297364' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T15:30:03.856 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:03 smithi096 ceph-mon[31553]: from='client.? 172.21.15.96:0/496297364' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0cd1f1d700 1 Processor -- start 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0cd1f1d700 1 -- start start 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0cd1f1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0cd1f1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc07be70 con 0x7f0ccc1087f0 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58088/0 (socket says 172.21.15.96:58088) 2024-03-18T15:30:04.761 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.147+0000 7f0ccb7fe700 1 -- 172.21.15.96:0/4065282809 learned_addr learned my addr 172.21.15.96:0/4065282809 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.148+0000 7f0ccb7fe700 1 -- 172.21.15.96:0/4065282809 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc10b120 con 0x7f0ccc1087f0 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.148+0000 7f0ccb7fe700 1 --2- 172.21.15.96:0/4065282809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0098a0 tx=0x7f0cbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a331fd4bc1295b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.148+0000 7f0cca7fc700 1 -- 172.21.15.96:0/4065282809 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc004160 con 0x7f0ccc1087f0 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.148+0000 7f0cca7fc700 1 -- 172.21.15.96:0/4065282809 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cbc004300 con 0x7f0ccc1087f0 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.148+0000 7f0cca7fc700 1 -- 172.21.15.96:0/4065282809 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc011b50 con 0x7f0ccc1087f0 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/4065282809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 msgr2=0x7f0ccc10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 --2- 172.21.15.96:0/4065282809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0098a0 tx=0x7f0cbc009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/4065282809 shutdown_connections 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 --2- 172.21.15.96:0/4065282809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc10abe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/4065282809 >> 172.21.15.96:0/4065282809 conn(0x7f0ccc102220 msgr2=0x7f0ccc104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/4065282809 shutdown_connections 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/4065282809 wait complete. 2024-03-18T15:30:04.762 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 Processor -- start 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.149+0000 7f0cd1f1d700 1 -- start start 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cd1f1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cd1f1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10c140 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0ccb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58090/0 (socket says 172.21.15.96:58090) 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0ccb7fe700 1 -- 172.21.15.96:0/1473073794 learned_addr learned my addr 172.21.15.96:0/1473073794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0ccb7fe700 1 -- 172.21.15.96:0/1473073794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc19c540 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0ccb7fe700 1 --2- 172.21.15.96:0/1473073794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc011670 tx=0x7f0cbc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc0039c0 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc19c7d0 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0cbc011e50 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc01a5c0 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc19f470 con 0x7f0ccc1087f0 2024-03-18T15:30:04.763 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.150+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0cbc01ac50 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.151+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0cbc019700 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.151+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc063900 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.153+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0cbc003b60 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.221+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0ccc02db70 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cc8ff9700 1 -- 172.21.15.96:0/1473073794 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0cbc0199a0 con 0x7f0ccc1087f0 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 msgr2=0x7f0ccc19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cd1f1d700 1 --2- 172.21.15.96:0/1473073794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc011670 tx=0x7f0cbc011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 shutdown_connections 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cd1f1d700 1 --2- 172.21.15.96:0/1473073794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ccc1087f0 0x7f0ccc19c000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.222+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 >> 172.21.15.96:0/1473073794 conn(0x7f0ccc102220 msgr2=0x7f0ccc102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.223+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 shutdown_connections 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:04.223+0000 7f0cd1f1d700 1 -- 172.21.15.96:0/1473073794 wait complete. 2024-03-18T15:30:04.764 INFO:teuthology.orchestra.run.smithi096.stdout:Restarting the monitor... 2024-03-18T15:30:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:04 smithi096 ceph-mon[31553]: from='client.? 172.21.15.96:0/1473073794' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:30:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:04 smithi096 systemd[1]: Stopping Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:30:05.347 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a[31549]: 2024-03-18T15:30:05.023+0000 7fd6cc3ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:30:05.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a[31549]: 2024-03-18T15:30:05.023+0000 7fd6cc3ce700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-18T15:30:05.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 podman[31802]: 2024-03-18 15:30:05.034144444 +0000 UTC m=+0.122620298 container died b9728c0c4693fbeaf66397ceddf1aef8a7af6482fa836ef6cfada93a52665012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:30:05.348 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 podman[31802]: 2024-03-18 15:30:05.347908904 +0000 UTC m=+0.436384756 container cleanup b9728c0c4693fbeaf66397ceddf1aef8a7af6482fa836ef6cfada93a52665012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-18T15:30:05.619 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 bash[31802]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a 2024-03-18T15:30:05.619 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 podman[31816]: 2024-03-18 15:30:05.507280435 +0000 UTC m=+0.471945006 container remove b9728c0c4693fbeaf66397ceddf1aef8a7af6482fa836ef6cfada93a52665012 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:30:05.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-18T15:30:05.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 systemd[1]: Stopped Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:30:05.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 systemd[1]: Starting Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:30:05.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 podman[31911]: 2024-03-18 15:30:05.849764027 +0000 UTC m=+0.110375563 container create 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-18T15:30:05.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:05 smithi096 podman[31911]: 2024-03-18 15:30:05.756981979 +0000 UTC m=+0.017593517 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-18T15:30:06.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 podman[31911]: 2024-03-18 15:30:06.01882095 +0000 UTC m=+0.279432479 container init 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:30:06.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 podman[31911]: 2024-03-18 15:30:06.021703504 +0000 UTC m=+0.282315035 container start 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: pidfile_write: ignore empty --pid-file 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: load: jerasure load: lrc 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: RocksDB version: 7.9.2 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Git sha 0 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: DB SUMMARY 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: DB Session ID: EN8BER7LSPQVOE9M92E7 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: CURRENT file: CURRENT 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T15:30:06.305 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104888 ; 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.error_if_exists: 0 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.create_if_missing: 0 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.paranoid_checks: 1 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.env: 0x56166f2abc60 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.info_log: 0x56166ff7dac0 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T15:30:06.306 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.statistics: (nil) 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.use_fsync: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_log_file_size: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_fallocate: 1 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.use_direct_reads: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.db_log_dir: 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.wal_dir: 2024-03-18T15:30:06.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.write_buffer_manager: 0x56166ff501e0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T15:30:06.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.unordered_write: 0 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.row_cache: None 2024-03-18T15:30:06.309 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.wal_filter: None 2024-03-18T15:30:06.310 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T15:30:06.310 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.two_write_queues: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.wal_compression: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.atomic_flush: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.log_readahead_size: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T15:30:06.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_background_jobs: 2 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_background_compactions: -1 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_subcompactions: 1 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_open_files: -1 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T15:30:06.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_background_flushes: -1 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Compression algorithms supported: 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kZSTD supported: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kXpressCompression supported: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kZlibCompression supported: 1 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kSnappyCompression supported: 1 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kLZ4Compression supported: 1 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: kBZip2Compression supported: 0 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-18T15:30:06.313 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.merge_operator: 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_filter: None 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56166ff7dbc0) 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_type: 0 2024-03-18T15:30:06.314 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_index_type: 0 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_shortening: 1 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: checksum: 4 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: no_block_cache: 0 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache: 0x56166ff83090 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_options: 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: capacity : 536870912 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_shard_bits : 4 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: strict_capacity_limit : 0 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_compressed: (nil) 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: persistent_cache: (nil) 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size: 4096 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size_deviation: 10 2024-03-18T15:30:06.315 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_restart_interval: 16 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_block_restart_interval: 1 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: metadata_block_size: 4096 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: partition_filters: 0 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: use_delta_encoding: 1 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: filter_policy: bloomfilter 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: whole_key_filtering: 1 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: verify_compression: 0 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: format_version: 5 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_index_compression: 1 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_align: 0 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: max_auto_readahead_size: 262144 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: prepopulate_block_cache: 0 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: initial_auto_readahead_size: 8192 2024-03-18T15:30:06.316 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression: NoCompression 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.num_levels: 7 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T15:30:06.317 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T15:30:06.318 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T15:30:06.319 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T15:30:06.320 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.table_properties_collectors: 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.inplace_update_support: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.bloom_locality: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.max_successive_merges: 0 2024-03-18T15:30:06.321 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.ttl: 2592000 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enable_blob_files: false 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.min_blob_size: 0 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T15:30:06.322 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2c17a0cf-e9f8-4b0e-9741-a5715f92a947 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775806050111, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775806051092, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97949, "index_size": 306, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14108, "raw_average_key_size": 49, "raw_value_size": 90275, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710775806, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2c17a0cf-e9f8-4b0e-9741-a5715f92a947", "db_session_id": "EN8BER7LSPQVOE9M92E7", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-18T15:30:06.323 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775806051174, "job": 1, "event": "recovery_finished"} 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561670076000 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: DB pointer 0x561670060000 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** DB Stats ** 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T15:30:06.324 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: L0 2/0 99.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: Sum 2/0 99.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T15:30:06.325 INFO:journalctl@ceph.mon.a.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative compaction: 0.00 GB write, 33.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T15:30:06.326 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval compaction: 0.00 GB write, 33.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache BinnedLRUCache@0x56166ff83090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: starting mon.a rank 0 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???) e1 preinit fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).mds e1 new map 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).mds e1 print_map 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: e1 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:30:06.327 INFO:journalctl@ceph.mon.a.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout: legacy client fscid: -1 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout: No filesystems configured 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 bash[31911]: 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 systemd[1]: Started Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-18T15:30:06.328 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: monmap epoch 1 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: last_changed 2024-03-18T15:29:58.578642+0000 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: min_mon_release 19 (squid) 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: election_strategy: 1 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: fsmap 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: osdmap e1: 0 total, 0 up, 0 in 2024-03-18T15:30:06.329 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:06 smithi096 ceph-mon[31930]: mgrmap e1: no daemons active 2024-03-18T15:30:07.153 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa72a761700 1 Processor -- start 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa72a761700 1 -- start start 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa72a761700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa72a761700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa724107630 con 0x7fa724106c40 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa723fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa723fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58100/0 (socket says 172.21.15.96:58100) 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.506+0000 7fa723fff700 1 -- 172.21.15.96:0/621861764 learned_addr learned my addr 172.21.15.96:0/621861764 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.507+0000 7fa723fff700 1 -- 172.21.15.96:0/621861764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa724107e40 con 0x7fa724106c40 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.507+0000 7fa723fff700 1 --2- 172.21.15.96:0/621861764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7140098a0 tx=0x7fa714009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08062a4379a383c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:07.158 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.507+0000 7fa722ffd700 1 -- 172.21.15.96:0/621861764 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa714004160 con 0x7fa724106c40 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.507+0000 7fa722ffd700 1 -- 172.21.15.96:0/621861764 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa714004300 con 0x7fa724106c40 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.507+0000 7fa722ffd700 1 -- 172.21.15.96:0/621861764 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa714011b50 con 0x7fa724106c40 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- 172.21.15.96:0/621861764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 msgr2=0x7fa724107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 --2- 172.21.15.96:0/621861764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa7140098a0 tx=0x7fa714009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- 172.21.15.96:0/621861764 shutdown_connections 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 --2- 172.21.15.96:0/621861764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa724107060 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- 172.21.15.96:0/621861764 >> 172.21.15.96:0/621861764 conn(0x7fa724102220 msgr2=0x7fa724104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- 172.21.15.96:0/621861764 shutdown_connections 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- 172.21.15.96:0/621861764 wait complete. 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 Processor -- start 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.508+0000 7fa72a761700 1 -- start start 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa72a761700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa72a761700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410b1f0 con 0x7fa724106c40 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa723fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:07.159 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa723fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58116/0 (socket says 172.21.15.96:58116) 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa723fff700 1 -- 172.21.15.96:0/1653487483 learned_addr learned my addr 172.21.15.96:0/1653487483 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa723fff700 1 -- 172.21.15.96:0/1653487483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72419b460 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa723fff700 1 --2- 172.21.15.96:0/1653487483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa714011670 tx=0x7fa714011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7140039c0 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa714011e50 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72419b6f0 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa71401a5c0 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.509+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72419e390 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.510+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa71401ac00 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.510+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa714019700 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.510+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724063900 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.513+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa714003b60 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.582+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa72402db70 con 0x7fa724106c40 2024-03-18T15:30:07.160 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.584+0000 7fa7217fa700 1 -- 172.21.15.96:0/1653487483 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa714019900 con 0x7fa724106c40 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 msgr2=0x7fa72419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 --2- 172.21.15.96:0/1653487483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa714011670 tx=0x7fa714011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 shutdown_connections 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 --2- 172.21.15.96:0/1653487483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa724106c40 0x7fa72419af20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 >> 172.21.15.96:0/1653487483 conn(0x7fa724102220 msgr2=0x7fa724102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 shutdown_connections 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:06.585+0000 7fa72a761700 1 -- 172.21.15.96:0/1653487483 wait complete. 2024-03-18T15:30:07.161 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-18T15:30:07.183 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:30:07.183 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mgr... 2024-03-18T15:30:07.183 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-18T15:30:07.184 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-18T15:30:07.427 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x 2024-03-18T15:30:07.427 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x.service: Unit ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x.service not loaded. 2024-03-18T15:30:07.595 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715.target.wants/ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-419c928c-e53c-11ee-95c9-87774f69a715@.service. 2024-03-18T15:30:07.921 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:07 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1653487483' entity='client.admin' 2024-03-18T15:30:08.117 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T15:30:08.117 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-03-18T15:30:08.124 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T15:30:08.124 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:30:08.124 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-03-18T15:30:08.124 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-03-18T15:30:08.132 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T15:30:08.132 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-03-18T15:30:08.140 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T15:30:08.141 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:30:08.141 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-03-18T15:30:08.141 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-18T15:30:08.141 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr to start... 2024-03-18T15:30:08.141 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr... 2024-03-18T15:30:08.918 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:08 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1689354215' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T15:30:09.173 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:09.173 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:09.173 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:30:09.173 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T15:30:09.174 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T15:30:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T15:30:09.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T15:30:09.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T15:30:09.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T15:30:09.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T15:30:01.553612+0000", 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T15:30:09.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0357388700 1 Processor -- start 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0357388700 1 -- start start 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0357388700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0357388700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0350075a00 con 0x7f0350108700 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0355124700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.565+0000 7f0355124700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58152/0 (socket says 172.21.15.96:58152) 2024-03-18T15:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0355124700 1 -- 172.21.15.96:0/3617304907 learned_addr learned my addr 172.21.15.96:0/3617304907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0355124700 1 -- 172.21.15.96:0/3617304907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035010b030 con 0x7f0350108700 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0355124700 1 --2- 172.21.15.96:0/3617304907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af2f8d09f8f3cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0347fff700 1 -- 172.21.15.96:0/3617304907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c004160 con 0x7f0350108700 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0347fff700 1 -- 172.21.15.96:0/3617304907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f034c004300 con 0x7f0350108700 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.566+0000 7f0347fff700 1 -- 172.21.15.96:0/3617304907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c011b50 con 0x7f0350108700 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 -- 172.21.15.96:0/3617304907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 msgr2=0x7f035010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 --2- 172.21.15.96:0/3617304907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f034c0098a0 tx=0x7f034c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 -- 172.21.15.96:0/3617304907 shutdown_connections 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 --2- 172.21.15.96:0/3617304907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035010aaf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 -- 172.21.15.96:0/3617304907 >> 172.21.15.96:0/3617304907 conn(0x7f0350102130 msgr2=0x7f0350104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 -- 172.21.15.96:0/3617304907 shutdown_connections 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.567+0000 7f0357388700 1 -- 172.21.15.96:0/3617304907 wait complete. 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0357388700 1 Processor -- start 2024-03-18T15:30:09.192 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0357388700 1 -- start start 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0357388700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0357388700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f035010c050 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0355124700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0355124700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58164/0 (socket says 172.21.15.96:58164) 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0355124700 1 -- 172.21.15.96:0/1689354215 learned_addr learned my addr 172.21.15.96:0/1689354215 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0355124700 1 -- 172.21.15.96:0/1689354215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035019c120 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f0355124700 1 --2- 172.21.15.96:0/1689354215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f034c011670 tx=0x7f034c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c0039c0 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.568+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f034c011e50 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034c01a5c0 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035019c3b0 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035019f0a0 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f034c01ac00 con 0x7f0350108700 2024-03-18T15:30:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f034c019700 con 0x7f0350108700 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.569+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350063900 con 0x7f0350108700 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.572+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f034c003b60 con 0x7f0350108700 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.643+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f035019f270 con 0x7f0350108700 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.644+0000 7f03467fc700 1 -- 172.21.15.96:0/1689354215 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f034c0199a0 con 0x7f0350108700 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 msgr2=0x7f035019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 --2- 172.21.15.96:0/1689354215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f034c011670 tx=0x7f034c011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:09.194 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 shutdown_connections 2024-03-18T15:30:09.195 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 --2- 172.21.15.96:0/1689354215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0350108700 0x7f035019bbe0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:09.195 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 >> 172.21.15.96:0/1689354215 conn(0x7f0350102130 msgr2=0x7f0350102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:09.195 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 shutdown_connections 2024-03-18T15:30:09.195 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:08.645+0000 7f0357388700 1 -- 172.21.15.96:0/1689354215 wait complete. 2024-03-18T15:30:09.195 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (1/15)... 2024-03-18T15:30:09.421 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.330+0000 7f7716290200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:30:09.421 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.420+0000 7f7716290200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:30:09.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.560+0000 7f7716290200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:30:09.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.654+0000 7f7716290200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:30:09.995 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.736+0000 7f7716290200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:30:09.995 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:09.994+0000 7f7716290200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T15:30:10.282 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:10.103+0000 7f7716290200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:10.283 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:10.188+0000 7f7716290200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:30:10.283 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:10.281+0000 7f7716290200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:30:10.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:10.454+0000 7f7716290200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:30:11.560 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:11 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:11.423+0000 7f7716290200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T15:30:11.560 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:11 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:11.559+0000 7f7716290200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:30:11.923 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:11 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/150267782' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T15:30:12.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:11 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:11.922+0000 7f7716290200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:30:12.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:12.013+0000 7f7716290200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:30:12.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:12.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:12.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:30:12.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T15:30:12.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:12.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T15:30:12.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T15:30:12.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T15:30:01.553612+0000", 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:12.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e72f39700 1 Processor -- start 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e72f39700 1 -- start start 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e72f39700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e72f39700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c07b800 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e70cd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e70cd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58172/0 (socket says 172.21.15.96:58172) 2024-03-18T15:30:12.337 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e70cd5700 1 -- 172.21.15.96:0/4204962850 learned_addr learned my addr 172.21.15.96:0/4204962850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.615+0000 7f9e70cd5700 1 -- 172.21.15.96:0/4204962850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c07b980 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e70cd5700 1 --2- 172.21.15.96:0/4204962850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e80 tx=0x7f9e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254004c106f0cc4c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e6b7fe700 1 -- 172.21.15.96:0/4204962850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64017070 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e6b7fe700 1 -- 172.21.15.96:0/4204962850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e6400b8c0 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e6b7fe700 1 -- 172.21.15.96:0/4204962850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e640045c0 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e72f39700 1 -- 172.21.15.96:0/4204962850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 msgr2=0x7f9e6c07b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.616+0000 7f9e72f39700 1 --2- 172.21.15.96:0/4204962850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009e80 tx=0x7f9e6400b330 comp rx=0 tx=0).stop 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- 172.21.15.96:0/4204962850 shutdown_connections 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 --2- 172.21.15.96:0/4204962850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c07b230 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- 172.21.15.96:0/4204962850 >> 172.21.15.96:0/4204962850 conn(0x7f9e6c103240 msgr2=0x7f9e6c105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- 172.21.15.96:0/4204962850 shutdown_connections 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- 172.21.15.96:0/4204962850 wait complete. 2024-03-18T15:30:12.338 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 Processor -- start 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- start start 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.617+0000 7f9e72f39700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c10e2a0 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e70cd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e70cd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58176/0 (socket says 172.21.15.96:58176) 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e70cd5700 1 -- 172.21.15.96:0/150267782 learned_addr learned my addr 172.21.15.96:0/150267782 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e70cd5700 1 -- 172.21.15.96:0/150267782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c19e370 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e70cd5700 1 --2- 172.21.15.96:0/150267782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e640072a0 tx=0x7f9e6400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64020020 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6c19e600 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e64003b50 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e64017040 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6c1a12f0 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.339 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e64023020 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.618+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e6400f050 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.619+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e6c063900 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.622+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9e6402a020 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.719+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9e6c1a1890 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.719+0000 7f9e69ffb700 1 -- 172.21.15.96:0/150267782 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9e6400b860 con 0x7f9e6c07cdd0 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 msgr2=0x7f9e6c19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 --2- 172.21.15.96:0/150267782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e640072a0 tx=0x7f9e6400bd50 comp rx=0 tx=0).stop 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 shutdown_connections 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 --2- 172.21.15.96:0/150267782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e6c07cdd0 0x7f9e6c19de30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 >> 172.21.15.96:0/150267782 conn(0x7f9e6c103240 msgr2=0x7f9e6c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 shutdown_connections 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:11.720+0000 7f9e72f39700 1 -- 172.21.15.96:0/150267782 wait complete. 2024-03-18T15:30:12.340 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (2/15)... 2024-03-18T15:30:12.678 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:12.210+0000 7f7716290200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:12.679 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:12.397+0000 7f7716290200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:30:12.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:12.678+0000 7f7716290200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:13.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:12.957+0000 7f7716290200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:13.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:13 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:13.190+0000 7f7716290200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:30:14.436 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:13 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:13.996+0000 7f7716290200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:30:14.710 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:14 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:14.578+0000 7f7716290200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:30:14.711 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:14 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:14.676+0000 7f7716290200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:30:15.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:14 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1440731155' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T15:30:15.187 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:14 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:14.915+0000 7f7716290200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:30:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:30:15.508 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T15:30:15.509 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:15.510 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T15:30:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T15:30:15.512 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T15:30:01.553612+0000", 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T15:30:15.513 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef3d390700 1 Processor -- start 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef3d390700 1 -- start start 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef3d390700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef3d390700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef3807be70 con 0x7fef38108710 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38722/0 (socket says 172.21.15.96:38722) 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.725+0000 7fef36ffd700 1 -- 172.21.15.96:0/2842678194 learned_addr learned my addr 172.21.15.96:0/2842678194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.726+0000 7fef36ffd700 1 -- 172.21.15.96:0/2842678194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3810b040 con 0x7fef38108710 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.726+0000 7fef36ffd700 1 --2- 172.21.15.96:0/2842678194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0098a0 tx=0x7fef2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a37e07e2c0a09cd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:15.514 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.726+0000 7fef35ffb700 1 -- 172.21.15.96:0/2842678194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef2c004160 con 0x7fef38108710 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.726+0000 7fef35ffb700 1 -- 172.21.15.96:0/2842678194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef2c004300 con 0x7fef38108710 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.726+0000 7fef35ffb700 1 -- 172.21.15.96:0/2842678194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef2c011b50 con 0x7fef38108710 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 -- 172.21.15.96:0/2842678194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 msgr2=0x7fef3810ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 --2- 172.21.15.96:0/2842678194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fef2c0098a0 tx=0x7fef2c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 -- 172.21.15.96:0/2842678194 shutdown_connections 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 --2- 172.21.15.96:0/2842678194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3810ab00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 -- 172.21.15.96:0/2842678194 >> 172.21.15.96:0/2842678194 conn(0x7fef381020e0 msgr2=0x7fef38104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 -- 172.21.15.96:0/2842678194 shutdown_connections 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 -- 172.21.15.96:0/2842678194 wait complete. 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.727+0000 7fef3d390700 1 Processor -- start 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef3d390700 1 -- start start 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef3d390700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef3d390700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef3810c060 con 0x7fef38108710 2024-03-18T15:30:15.515 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef36ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38724/0 (socket says 172.21.15.96:38724) 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef36ffd700 1 -- 172.21.15.96:0/1440731155 learned_addr learned my addr 172.21.15.96:0/1440731155 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef36ffd700 1 -- 172.21.15.96:0/1440731155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3819c170 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.728+0000 7fef36ffd700 1 --2- 172.21.15.96:0/1440731155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2c011670 tx=0x7fef2c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef2c0039c0 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef2c011e50 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3819c400 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef2c01a5c0 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3819f0f0 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef2c01ac00 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.729+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fef2c019700 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.730+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef38063900 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.732+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fef2c003b60 con 0x7fef38108710 2024-03-18T15:30:15.516 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.823+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fef3819f2c0 con 0x7fef38108710 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef1ffff700 1 -- 172.21.15.96:0/1440731155 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fef2c0199a0 con 0x7fef38108710 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 msgr2=0x7fef3819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef3d390700 1 --2- 172.21.15.96:0/1440731155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fef2c011670 tx=0x7fef2c011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 shutdown_connections 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef3d390700 1 --2- 172.21.15.96:0/1440731155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef38108710 0x7fef3819bc30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.824+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 >> 172.21.15.96:0/1440731155 conn(0x7fef381020e0 msgr2=0x7fef38102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.825+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 shutdown_connections 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:14.825+0000 7fef3d390700 1 -- 172.21.15.96:0/1440731155 wait complete. 2024-03-18T15:30:15.517 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (3/15)... 2024-03-18T15:30:16.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: Activating manager daemon x 2024-03-18T15:30:16.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: mgrmap e2: x(active, starting, since 0.00315926s) 2024-03-18T15:30:16.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T15:30:16.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: Manager daemon x is now available 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' 2024-03-18T15:30:16.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:15 smithi096 ceph-mon[31930]: from='mgr.14100 172.21.15.96:0/504377701' entity='mgr.x' 2024-03-18T15:30:17.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:16 smithi096 ceph-mon[31930]: mgrmap e3: x(active, since 1.0074s) 2024-03-18T15:30:19.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:18 smithi096 ceph-mon[31930]: mgrmap e4: x(active, since 3s) 2024-03-18T15:30:21.068 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:20 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4118084405' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-18T15:30:26.163 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-18T15:30:26.164 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-18T15:30:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-18T15:30:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-18T15:30:26.165 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-18T15:30:26.178 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-18T15:30:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-18T15:30:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-18T15:30:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-18T15:30:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-18T15:30:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-18T15:30:26.181 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-03-18T15:30:01.553612+0000", 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.422+0000 7f364ff50700 1 Processor -- start 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.422+0000 7f364ff50700 1 -- start start 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364ff50700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364ff50700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f364807be50 con 0x7f3648106560 2024-03-18T15:30:26.182 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364dcec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364dcec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38800/0 (socket says 172.21.15.96:38800) 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364dcec700 1 -- 172.21.15.96:0/886837863 learned_addr learned my addr 172.21.15.96:0/886837863 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364dcec700 1 -- 172.21.15.96:0/886837863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3648106ec0 con 0x7f3648106560 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364dcec700 1 --2- 172.21.15.96:0/886837863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e80 tx=0x7f364000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9494c8bd95bd633a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364ccea700 1 -- 172.21.15.96:0/886837863 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3640017070 con 0x7f3648106560 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364ccea700 1 -- 172.21.15.96:0/886837863 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f364000b8c0 con 0x7f3648106560 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.423+0000 7f364ccea700 1 -- 172.21.15.96:0/886837863 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36400045c0 con 0x7f3648106560 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 -- 172.21.15.96:0/886837863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 msgr2=0x7f3648106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 --2- 172.21.15.96:0/886837863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3640009e80 tx=0x7f364000b330 comp rx=0 tx=0).stop 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 -- 172.21.15.96:0/886837863 shutdown_connections 2024-03-18T15:30:26.183 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 --2- 172.21.15.96:0/886837863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f3648106980 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 -- 172.21.15.96:0/886837863 >> 172.21.15.96:0/886837863 conn(0x7f3648102120 msgr2=0x7f3648104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 -- 172.21.15.96:0/886837863 shutdown_connections 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.424+0000 7f364ff50700 1 -- 172.21.15.96:0/886837863 wait complete. 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364ff50700 1 Processor -- start 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364ff50700 1 -- start start 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364ff50700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364ff50700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3648107f50 con 0x7f3648106560 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364dcec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364dcec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38816/0 (socket says 172.21.15.96:38816) 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364dcec700 1 -- 172.21.15.96:0/4118084405 learned_addr learned my addr 172.21.15.96:0/4118084405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364dcec700 1 -- 172.21.15.96:0/4118084405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364819e610 con 0x7f3648106560 2024-03-18T15:30:26.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f364dcec700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f36400072a0 tx=0x7f364000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.425+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3640020020 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364819e8a0 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a1590 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3640003b50 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3640017040 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f3640023020 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.426+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3634005360 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.427+0000 7f3646ffd700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 0x7f363003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.427+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f364000f050 con 0x7f3648106560 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.427+0000 7f364d4eb700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 0x7f363003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.427+0000 7f364d4eb700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 0x7f363003b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f363c006fd0 tx=0x7f363c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:26.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.432+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f364002a020 con 0x7f3648106560 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.710+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3634005650 con 0x7f3648106560 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.710+0000 7f3646ffd700 1 -- 172.21.15.96:0/4118084405 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3640018240 con 0x7f3648106560 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 msgr2=0x7f363003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 0x7f363003b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f363c006fd0 tx=0x7f363c006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 msgr2=0x7f364819e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f36400072a0 tx=0x7f364000bd50 comp rx=0 tx=0).stop 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 shutdown_connections 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f3630038ed0 0x7f363003b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 --2- 172.21.15.96:0/4118084405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3648106560 0x7f364819e0d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 >> 172.21.15.96:0/4118084405 conn(0x7f3648102120 msgr2=0x7f3648194a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:26.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 shutdown_connections 2024-03-18T15:30:26.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:20.713+0000 7f364ff50700 1 -- 172.21.15.96:0/4118084405 wait complete. 2024-03-18T15:30:26.187 INFO:teuthology.orchestra.run.smithi096.stdout:mgr is available 2024-03-18T15:30:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:26 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1267550829' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-18T15:30:27.331 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:27.331 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-03-18T15:30:27.331 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-18T15:30:27.332 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb76862f700 1 Processor -- start 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb76862f700 1 -- start start 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb76862f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb76862f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb760108b30 con 0x7fb760108140 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb7663cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb7663cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37336/0 (socket says 172.21.15.96:37336) 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.581+0000 7fb7663cb700 1 -- 172.21.15.96:0/4133090285 learned_addr learned my addr 172.21.15.96:0/4133090285 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb7663cb700 1 -- 172.21.15.96:0/4133090285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb760106da0 con 0x7fb760108140 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb7663cb700 1 --2- 172.21.15.96:0/4133090285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd40a3b2c89e37f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb7653c9700 1 -- 172.21.15.96:0/4133090285 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c004160 con 0x7fb760108140 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb7653c9700 1 -- 172.21.15.96:0/4133090285 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c004300 con 0x7fb760108140 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb7653c9700 1 -- 172.21.15.96:0/4133090285 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c011b50 con 0x7fb760108140 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb76862f700 1 -- 172.21.15.96:0/4133090285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 msgr2=0x7fb760108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.582+0000 7fb76862f700 1 --2- 172.21.15.96:0/4133090285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0098a0 tx=0x7fb75c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- 172.21.15.96:0/4133090285 shutdown_connections 2024-03-18T15:30:27.333 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 --2- 172.21.15.96:0/4133090285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760108560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- 172.21.15.96:0/4133090285 >> 172.21.15.96:0/4133090285 conn(0x7fb7601018d0 msgr2=0x7fb760103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- 172.21.15.96:0/4133090285 shutdown_connections 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- 172.21.15.96:0/4133090285 wait complete. 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 Processor -- start 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- start start 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb76862f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb76010b160 con 0x7fb760108140 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.583+0000 7fb7663cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7663cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37340/0 (socket says 172.21.15.96:37340) 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7663cb700 1 -- 172.21.15.96:0/1267550829 learned_addr learned my addr 172.21.15.96:0/1267550829 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7663cb700 1 -- 172.21.15.96:0/1267550829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76019e970 con 0x7fb760108140 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7663cb700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011670 tx=0x7fb75c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:27.334 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c0039c0 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb75c011e50 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76019eb40 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75c01a5c0 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.584+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76019eee0 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7fb75c01a8c0 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb760063900 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb7577fe700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 0x7fb74c03b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb75c04d3d0 con 0x7fb760108140 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb765bca700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 0x7fb74c03b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.585+0000 7fb765bca700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 0x7fb74c03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:27.335 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.590+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb75c019c70 con 0x7fb760108140 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.783+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb76019f160 con 0x7fb760108140 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.784+0000 7fb7577fe700 1 -- 172.21.15.96:0/1267550829 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fb75c0186d0 con 0x7fb760108140 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 msgr2=0x7fb74c03b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 0x7fb74c03b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb750006fd0 tx=0x7fb750006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 msgr2=0x7fb760106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb75c011670 tx=0x7fb75c011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 shutdown_connections 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fb74c038e80 0x7fb74c03b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 --2- 172.21.15.96:0/1267550829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb760108140 0x7fb760106710 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 >> 172.21.15.96:0/1267550829 conn(0x7fb7601018d0 msgr2=0x7fb760102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 shutdown_connections 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:26.786+0000 7fb76862f700 1 -- 172.21.15.96:0/1267550829 wait complete. 2024-03-18T15:30:27.336 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling cephadm module... 2024-03-18T15:30:28.361 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2428846990' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-18T15:30:28.361 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: ignoring --setuser ceph since I am not root 2024-03-18T15:30:28.362 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: ignoring --setgroup ceph since I am not root 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f7770142700 1 Processor -- start 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f7770142700 1 -- start start 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f7770142700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f7770142700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7768106920 con 0x7f7768105f30 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f776dede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f776dede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37346/0 (socket says 172.21.15.96:37346) 2024-03-18T15:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.731+0000 7f776dede700 1 -- 172.21.15.96:0/2625661591 learned_addr learned my addr 172.21.15.96:0/2625661591 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f776dede700 1 -- 172.21.15.96:0/2625661591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7768108ec0 con 0x7f7768105f30 2024-03-18T15:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f776dede700 1 --2- 172.21.15.96:0/2625661591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77640098a0 tx=0x7f7764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f172536f4d88e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f776cedc700 1 -- 172.21.15.96:0/2625661591 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764004160 con 0x7f7768105f30 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f776cedc700 1 -- 172.21.15.96:0/2625661591 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7764004300 con 0x7f7768105f30 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f776cedc700 1 -- 172.21.15.96:0/2625661591 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7764011b50 con 0x7f7768105f30 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f7770142700 1 -- 172.21.15.96:0/2625661591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 msgr2=0x7f7768106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f7770142700 1 --2- 172.21.15.96:0/2625661591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77640098a0 tx=0x7f7764009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f7770142700 1 -- 172.21.15.96:0/2625661591 shutdown_connections 2024-03-18T15:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f7770142700 1 --2- 172.21.15.96:0/2625661591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f7768106350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.732+0000 7f7770142700 1 -- 172.21.15.96:0/2625661591 >> 172.21.15.96:0/2625661591 conn(0x7f7768101860 msgr2=0x7f7768103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 -- 172.21.15.96:0/2625661591 shutdown_connections 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 -- 172.21.15.96:0/2625661591 wait complete. 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 Processor -- start 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 -- start start 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f7770142700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f776810c270 con 0x7f7768105f30 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f776dede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f776dede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37360/0 (socket says 172.21.15.96:37360) 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.733+0000 7f776dede700 1 -- 172.21.15.96:0/2428846990 learned_addr learned my addr 172.21.15.96:0/2428846990 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f776dede700 1 -- 172.21.15.96:0/2428846990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776819c460 con 0x7f7768105f30 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f776dede700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7764011670 tx=0x7f7764011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77640039c0 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7764011e50 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776819c6f0 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f776401a5c0 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.734+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776819f390 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.735+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46095+0+0 (secure 0 0 0) 0x7f7764024430 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.735+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7768063900 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.735+0000 7f775effd700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 0x7f775403b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.735+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f776404d190 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.735+0000 7f776d6dd700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 0x7f775403b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.736+0000 7f776d6dd700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 0x7f775403b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7758006fd0 tx=0x7f7758006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.740+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f776404cbb0 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:27.990+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f776802db70 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.041+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f776402d3f0 con 0x7f7768105f30 2024-03-18T15:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.043+0000 7f775effd700 1 -- 172.21.15.96:0/2428846990 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f7764024720 con 0x7f7768105f30 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 msgr2=0x7f775403b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 0x7f775403b390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7758006fd0 tx=0x7f7758006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 msgr2=0x7f776819bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7764011670 tx=0x7f7764011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 shutdown_connections 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f7754038ed0 0x7f775403b390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 --2- 172.21.15.96:0/2428846990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7768105f30 0x7f776819bf20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 >> 172.21.15.96:0/2428846990 conn(0x7f7768101860 msgr2=0x7f7768102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 shutdown_connections 2024-03-18T15:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.044+0000 7f7770142700 1 -- 172.21.15.96:0/2428846990 wait complete. 2024-03-18T15:30:28.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:28.600+0000 7f2507157200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:30:28.964 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:28.698+0000 7f2507157200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:30:29.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2428846990' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-18T15:30:29.244 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-mon[31930]: mgrmap e5: x(active, since 13s) 2024-03-18T15:30:29.624 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:29.348+0000 7f2507157200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:30:29.625 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:29.440+0000 7f2507157200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:30:29.625 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:29.585+0000 7f2507157200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:30:29.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:29.679+0000 7f2507157200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:30:29.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:29.760+0000 7f2507157200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:30:30.294 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/636256111' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T15:30:30.294 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:30.019+0000 7f2507157200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T15:30:30.294 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:30.128+0000 7f2507157200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:30.295 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:30.212+0000 7f2507157200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T15:30:30.304 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe7a0a14700 1 Processor -- start 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe7a0a14700 1 -- start start 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe7a0a14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe7a0a14700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c07be50 con 0x7fe79c108730 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe79a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe79a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37388/0 (socket says 172.21.15.96:37388) 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.979+0000 7fe79a59c700 1 -- 172.21.15.96:0/1770541365 learned_addr learned my addr 172.21.15.96:0/1770541365 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe79a59c700 1 -- 172.21.15.96:0/1770541365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c10b060 con 0x7fe79c108730 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe79a59c700 1 --2- 172.21.15.96:0/1770541365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ee0b84e4b7c20a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe79959a700 1 -- 172.21.15.96:0/1770541365 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c004160 con 0x7fe79c108730 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe79959a700 1 -- 172.21.15.96:0/1770541365 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe78c004300 con 0x7fe79c108730 2024-03-18T15:30:30.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe79959a700 1 -- 172.21.15.96:0/1770541365 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c011b50 con 0x7fe79c108730 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe7a0a14700 1 -- 172.21.15.96:0/1770541365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 msgr2=0x7fe79c10ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.980+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/1770541365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- 172.21.15.96:0/1770541365 shutdown_connections 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/1770541365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c10ab20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- 172.21.15.96:0/1770541365 >> 172.21.15.96:0/1770541365 conn(0x7fe79c1020e0 msgr2=0x7fe79c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- 172.21.15.96:0/1770541365 shutdown_connections 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- 172.21.15.96:0/1770541365 wait complete. 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 Processor -- start 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- start start 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.981+0000 7fe7a0a14700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10c030 con 0x7fe79c108730 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe79a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe79a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37404/0 (socket says 172.21.15.96:37404) 2024-03-18T15:30:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe79a59c700 1 -- 172.21.15.96:0/636256111 learned_addr learned my addr 172.21.15.96:0/636256111 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe79a59c700 1 -- 172.21.15.96:0/636256111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c19c170 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe79a59c700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011b60 tx=0x7fe78c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c003b20 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c19c400 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe78c003cc0 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c024840 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.982+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c19f0f0 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.983+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c063900 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.983+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7fe78c022020 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.983+0000 7fe78b7fe700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 0x7fe78403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.984+0000 7fe799d9b700 1 -- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 msgr2=0x7fe78403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.984+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe78c011e00 con 0x7fe79c108730 2024-03-18T15:30:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.984+0000 7fe799d9b700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 0x7fe78403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:28.988+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe78c00ba00 con 0x7fe79c108730 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.185+0000 7fe799d9b700 1 -- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 msgr2=0x7fe78403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.185+0000 7fe799d9b700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 0x7fe78403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.242+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe79c19f2c0 con 0x7fe79c108730 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.243+0000 7fe78b7fe700 1 -- 172.21.15.96:0/636256111 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe79c19f2c0 con 0x7fe79c108730 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 msgr2=0x7fe78403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 0x7fe78403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 msgr2=0x7fe79c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011b60 tx=0x7fe78c011c40 comp rx=0 tx=0).stop 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 shutdown_connections 2024-03-18T15:30:30.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7fe784038f20 0x7fe78403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 --2- 172.21.15.96:0/636256111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe79c108730 0x7fe79c19bc30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.245+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 >> 172.21.15.96:0/636256111 conn(0x7fe79c1020e0 msgr2=0x7fe79c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.246+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 shutdown_connections 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:29.246+0000 7fe7a0a14700 1 -- 172.21.15.96:0/636256111 wait complete. 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-03-18T15:30:30.309 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 5... 2024-03-18T15:30:30.630 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:30.310+0000 7f2507157200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:30:30.630 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:30.491+0000 7f2507157200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:30:31.576 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:31.447+0000 7f2507157200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T15:30:31.905 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:31.575+0000 7f2507157200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:30:31.905 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:31.904+0000 7f2507157200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:30:32.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:31.993+0000 7f2507157200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:30:32.654 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:32.188+0000 7f2507157200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:32.654 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:32.371+0000 7f2507157200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:30:32.926 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:32.653+0000 7f2507157200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:32.926 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:32.924+0000 7f2507157200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:33.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:33.159+0000 7f2507157200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:30:34.438 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:33.956+0000 7f2507157200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:30:34.831 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:34.518+0000 7f2507157200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:30:34.832 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:34.615+0000 7f2507157200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:30:34.832 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:34.831+0000 7f2507157200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: Active manager daemon x restarted 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: Activating manager daemon x 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: osdmap e2: 0 total, 0 up, 0 in 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: mgrmap e6: x(active, starting, since 0.00337675s) 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:30:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: Manager daemon x is now available 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:35.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:34 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:35 smithi096 ceph-mon[31930]: Found migration_current of "None". Setting to last migration. 2024-03-18T15:30:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:35 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:35 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T15:30:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:35 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T15:30:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:35 smithi096 ceph-mon[31930]: mgrmap e7: x(active, since 1.00535s) 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87ce41d700 1 Processor -- start 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87ce41d700 1 -- start start 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87ce41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:36.393 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87ce41d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8075a00 con 0x7f87c8108700 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37414/0 (socket says 172.21.15.96:37414) 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.629+0000 7f87c7fff700 1 -- 172.21.15.96:0/1874538980 learned_addr learned my addr 172.21.15.96:0/1874538980 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.630+0000 7f87c7fff700 1 -- 172.21.15.96:0/1874538980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c810b030 con 0x7f87c8108700 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.630+0000 7f87c7fff700 1 --2- 172.21.15.96:0/1874538980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098a0 tx=0x7f87b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc1af1bea9bba29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.630+0000 7f87c6ffd700 1 -- 172.21.15.96:0/1874538980 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8004160 con 0x7f87c8108700 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.630+0000 7f87c6ffd700 1 -- 172.21.15.96:0/1874538980 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87b8004300 con 0x7f87c8108700 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.630+0000 7f87c6ffd700 1 -- 172.21.15.96:0/1874538980 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8011b50 con 0x7f87c8108700 2024-03-18T15:30:36.394 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- 172.21.15.96:0/1874538980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 msgr2=0x7f87c810aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 --2- 172.21.15.96:0/1874538980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f87b80098a0 tx=0x7f87b8009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- 172.21.15.96:0/1874538980 shutdown_connections 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 --2- 172.21.15.96:0/1874538980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c810aaf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- 172.21.15.96:0/1874538980 >> 172.21.15.96:0/1874538980 conn(0x7f87c8102130 msgr2=0x7f87c8104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- 172.21.15.96:0/1874538980 shutdown_connections 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- 172.21.15.96:0/1874538980 wait complete. 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 Processor -- start 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.631+0000 7f87ce41d700 1 -- start start 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87ce41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87ce41d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87c810c050 con 0x7f87c8108700 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37420/0 (socket says 172.21.15.96:37420) 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c7fff700 1 -- 172.21.15.96:0/2945433688 learned_addr learned my addr 172.21.15.96:0/2945433688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c7fff700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c819c120 con 0x7f87c8108700 2024-03-18T15:30:36.395 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c7fff700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87b8011b60 tx=0x7f87b8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8003b20 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87b8003cc0 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87b8024840 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.632+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c819c3b0 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.633+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c819f0a0 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.633+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 46106+0+0 (secure 0 0 0) 0x7f87b8022020 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.633+0000 7f87c57fa700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.633+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87b003baf0 con 0x7f87b0038f20 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.634+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f87b8011e00 con 0x7f87c8108700 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.634+0000 7f87c77fe700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.634+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.835+0000 7f87c77fe700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:30.835+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:30:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:31.236+0000 7f87c77fe700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:31.236+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:32.037+0000 7f87c77fe700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:32.037+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:33.638+0000 7f87c77fe700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/392915825 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:33.638+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:34.846+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 6) ==== 45873+0+0 (secure 0 0 0) 0x7f87b801a960 con 0x7f87c8108700 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:34.846+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 msgr2=0x7f87b003b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:34.846+0000 7f87c57fa700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.847+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f87b804eb90 con 0x7f87c8108700 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.847+0000 7f87c57fa700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 0x7f87b003ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.847+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f87b003baf0 con 0x7f87b003c810 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.847+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 0x7f87b003ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.848+0000 7f87c77fe700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 0x7f87b003ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f87bc003a10 tx=0x7f87bc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.848+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f87b003baf0 con 0x7f87b003c810 2024-03-18T15:30:36.397 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f87c8063900 con 0x7f87b003c810 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87c57fa700 1 -- 172.21.15.96:0/2945433688 <== mgr.14120 v2:172.21.15.96:6800/2602375712 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f87c8063900 con 0x7f87b003c810 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 msgr2=0x7f87b003ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 0x7f87b003ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f87bc003a10 tx=0x7f87bc009290 comp rx=0 tx=0).stop 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 msgr2=0x7f87c819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87b8011b60 tx=0x7f87b8011c40 comp rx=0 tx=0).stop 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 shutdown_connections 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f87b003c810 0x7f87b003ec00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c8108700 0x7f87c819bbe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 --2- 172.21.15.96:0/2945433688 >> [v2:172.21.15.96:6800/392915825,v1:172.21.15.96:6801/392915825] conn(0x7f87b0038f20 0x7f87b003b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 >> 172.21.15.96:0/2945433688 conn(0x7f87c8102130 msgr2=0x7f87c8102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 shutdown_connections 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:35.855+0000 7f87ce41d700 1 -- 172.21.15.96:0/2945433688 wait complete. 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 5 is available 2024-03-18T15:30:36.398 INFO:teuthology.orchestra.run.smithi096.stdout:Setting orchestrator backend to cephadm... 2024-03-18T15:30:37.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:36 smithi096 ceph-mon[31930]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T15:30:37.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:36 smithi096 ceph-mon[31930]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26adfb3700 1 Processor -- start 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26adfb3700 1 -- start start 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26adfb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26adfb3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81068d0 con 0x7f26a8105ee0 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50128/0 (socket says 172.21.15.96:50128) 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.725+0000 7f26a77fe700 1 -- 172.21.15.96:0/2858658593 learned_addr learned my addr 172.21.15.96:0/2858658593 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.726+0000 7f26a77fe700 1 -- 172.21.15.96:0/2858658593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a8106a50 con 0x7f26a8105ee0 2024-03-18T15:30:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.726+0000 7f26a77fe700 1 --2- 172.21.15.96:0/2858658593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f269c0098a0 tx=0x7f269c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11424a85e94c7017 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.726+0000 7f26a67fc700 1 -- 172.21.15.96:0/2858658593 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f269c004160 con 0x7f26a8105ee0 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.726+0000 7f26a67fc700 1 -- 172.21.15.96:0/2858658593 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f269c004300 con 0x7f26a8105ee0 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.726+0000 7f26a67fc700 1 -- 172.21.15.96:0/2858658593 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f269c011b50 con 0x7f26a8105ee0 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 -- 172.21.15.96:0/2858658593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 msgr2=0x7f26a8106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 --2- 172.21.15.96:0/2858658593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f269c0098a0 tx=0x7f269c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 -- 172.21.15.96:0/2858658593 shutdown_connections 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 --2- 172.21.15.96:0/2858658593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a8106300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 -- 172.21.15.96:0/2858658593 >> 172.21.15.96:0/2858658593 conn(0x7f26a81018a0 msgr2=0x7f26a8103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 -- 172.21.15.96:0/2858658593 shutdown_connections 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 -- 172.21.15.96:0/2858658593 wait complete. 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.727+0000 7f26adfb3700 1 Processor -- start 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26adfb3700 1 -- start start 2024-03-18T15:30:37.582 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26adfb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26adfb3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8109da0 con 0x7f26a8105ee0 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50132/0 (socket says 172.21.15.96:50132) 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a77fe700 1 -- 172.21.15.96:0/4075294350 learned_addr learned my addr 172.21.15.96:0/4075294350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a77fe700 1 -- 172.21.15.96:0/4075294350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a819c100 con 0x7f26a8105ee0 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a77fe700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f269c003b20 con 0x7f26a8105ee0 2024-03-18T15:30:37.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f269c003cc0 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a819c3f0 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.728+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f269c024910 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.729+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a819f080 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.729+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 7) ==== 46000+0+0 (secure 0 0 0) 0x7f269c022020 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.730+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a8063900 con 0x7f26a8105ee0 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.730+0000 7f26a4ff9700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 0x7f269003b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:37.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.730+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f269c011e00 con 0x7f26a8105ee0 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.730+0000 7f26a6ffd700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 0x7f269003b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.730+0000 7f26a6ffd700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 0x7f269003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2698006fd0 tx=0x7f2698006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.739+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f269c01acd0 con 0x7f26a8105ee0 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.899+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f269c081740 con 0x7f26a8105ee0 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.954+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f26a819f250 con 0x7f2690038e50 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.961+0000 7f26a4ff9700 1 -- 172.21.15.96:0/4075294350 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26a819f250 con 0x7f2690038e50 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.963+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 msgr2=0x7f269003b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.963+0000 7f26adfb3700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 0x7f269003b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2698006fd0 tx=0x7f2698006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.963+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 msgr2=0x7f26a819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:37.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.963+0000 7f26adfb3700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f269c000c00 tx=0x7f269c011c10 comp rx=0 tx=0).stop 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.963+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 shutdown_connections 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.964+0000 7f26adfb3700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f2690038e50 0x7f269003b310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.964+0000 7f26adfb3700 1 --2- 172.21.15.96:0/4075294350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26a8105ee0 0x7f26a819bbc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.964+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 >> 172.21.15.96:0/4075294350 conn(0x7f26a81018a0 msgr2=0x7f26a8102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.964+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 shutdown_connections 2024-03-18T15:30:37.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:36.964+0000 7f26adfb3700 1 -- 172.21.15.96:0/4075294350 wait complete. 2024-03-18T15:30:38.178 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: mgrmap e8: x(active, since 2s) 2024-03-18T15:30:38.178 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:38.178 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:38.178 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:38.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:38.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:38.179 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:37 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea3b24d700 1 Processor -- start 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea3b24d700 1 -- start start 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea3b24d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea3b24d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea34107940 con 0x7fea341093b0 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea38fe9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea38fe9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50134/0 (socket says 172.21.15.96:50134) 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea38fe9700 1 -- 172.21.15.96:0/1604233568 learned_addr learned my addr 172.21.15.96:0/1604233568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.973+0000 7fea38fe9700 1 -- 172.21.15.96:0/1604233568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea34107ac0 con 0x7fea341093b0 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea38fe9700 1 --2- 172.21.15.96:0/1604233568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea280098a0 tx=0x7fea28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ddd5cda488ab53 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea337fe700 1 -- 172.21.15.96:0/1604233568 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28004160 con 0x7fea341093b0 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea337fe700 1 -- 172.21.15.96:0/1604233568 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea28004300 con 0x7fea341093b0 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea337fe700 1 -- 172.21.15.96:0/1604233568 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28011b50 con 0x7fea341093b0 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 -- 172.21.15.96:0/1604233568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 msgr2=0x7fea341097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 --2- 172.21.15.96:0/1604233568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea280098a0 tx=0x7fea28009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 -- 172.21.15.96:0/1604233568 shutdown_connections 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 --2- 172.21.15.96:0/1604233568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea341097d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 -- 172.21.15.96:0/1604233568 >> 172.21.15.96:0/1604233568 conn(0x7fea34103320 msgr2=0x7fea34105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 -- 172.21.15.96:0/1604233568 shutdown_connections 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.974+0000 7fea3b24d700 1 -- 172.21.15.96:0/1604233568 wait complete. 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea3b24d700 1 Processor -- start 2024-03-18T15:30:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea3b24d700 1 -- start start 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea3b24d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea3b24d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea3410c280 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea38fe9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea38fe9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50142/0 (socket says 172.21.15.96:50142) 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea38fe9700 1 -- 172.21.15.96:0/3250253239 learned_addr learned my addr 172.21.15.96:0/3250253239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.975+0000 7fea38fe9700 1 -- 172.21.15.96:0/3250253239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419c220 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea38fe9700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28003b20 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea28003cc0 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea28024910 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3419c4b0 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.976+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3419f190 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.977+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fea28022020 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.977+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3410c400 con 0x7fea341093b0 2024-03-18T15:30:38.715 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.977+0000 7fea31ffb700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 0x7fea2403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.977+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea28011e00 con 0x7fea341093b0 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.977+0000 7fea33fff700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 0x7fea2403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.978+0000 7fea33fff700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 0x7fea2403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea20006fd0 tx=0x7fea20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:37.982+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea28016260 con 0x7fea341093b0 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.177+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fea34104590 con 0x7fea24038f20 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.177+0000 7fea31ffb700 1 -- 172.21.15.96:0/3250253239 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fea34104590 con 0x7fea24038f20 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 msgr2=0x7fea2403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 0x7fea2403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea20006fd0 tx=0x7fea20006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 msgr2=0x7fea3419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28011c10 comp rx=0 tx=0).stop 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 shutdown_connections 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fea24038f20 0x7fea2403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 --2- 172.21.15.96:0/3250253239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea341093b0 0x7fea3419bce0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:38.716 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 >> 172.21.15.96:0/3250253239 conn(0x7fea34103320 msgr2=0x7fea34103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:38.717 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 shutdown_connections 2024-03-18T15:30:38.717 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:38.180+0000 7fea3b24d700 1 -- 172.21.15.96:0/3250253239 wait complete. 2024-03-18T15:30:38.717 INFO:teuthology.orchestra.run.smithi096.stdout:Generating ssh key... 2024-03-18T15:30:39.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:38 smithi096 ceph-mon[31930]: [18/Mar/2024:15:30:37] ENGINE Bus STARTING 2024-03-18T15:30:39.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:38 smithi096 ceph-mon[31930]: [18/Mar/2024:15:30:37] ENGINE Serving on http://172.21.15.96:8765 2024-03-18T15:30:39.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:38 smithi096 ceph-mon[31930]: [18/Mar/2024:15:30:37] ENGINE Serving on https://172.21.15.96:7150 2024-03-18T15:30:39.082 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:38 smithi096 ceph-mon[31930]: [18/Mar/2024:15:30:37] ENGINE Bus STARTED 2024-03-18T15:30:39.810 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: Generating public/private rsa key pair. 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: Your identification has been saved in /tmp/tmp0htcflky/key. 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: Your public key has been saved in /tmp/tmp0htcflky/key.pub. 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: The key fingerprint is: 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: SHA256:roVKScw4AH96iZHKEWx+1tj/TsJ3eiwRt8ztL1kpnrg ceph-419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: The key's randomart image is: 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: +---[RSA 3072]----+ 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: |o. | 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: |.+.. | 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: |+.+ = | 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: |.+.% + . . | 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: |..B * . S = o .| 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: | + ..+ . + o o | 2024-03-18T15:30:39.811 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: | o .o+oo.+ = | 2024-03-18T15:30:39.812 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: | . . o+oo+ * | 2024-03-18T15:30:39.812 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: | . . .ooE. o. | 2024-03-18T15:30:39.812 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: +----[SHA256]-----+ 2024-03-18T15:30:40.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.096+0000 7f25f51e5700 1 Processor -- start 2024-03-18T15:30:40.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.096+0000 7f25f51e5700 1 -- start start 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.096+0000 7f25f51e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25f51e5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25f007be70 con 0x7f25f0108710 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25eed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25eed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50146/0 (socket says 172.21.15.96:50146) 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25eed9d700 1 -- 172.21.15.96:0/504454961 learned_addr learned my addr 172.21.15.96:0/504454961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25eed9d700 1 -- 172.21.15.96:0/504454961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f010b040 con 0x7f25f0108710 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25eed9d700 1 --2- 172.21.15.96:0/504454961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f25e40098a0 tx=0x7f25e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd405ead2510f5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25edd9b700 1 -- 172.21.15.96:0/504454961 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4004160 con 0x7f25f0108710 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25edd9b700 1 -- 172.21.15.96:0/504454961 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25e4004300 con 0x7f25f0108710 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.097+0000 7f25edd9b700 1 -- 172.21.15.96:0/504454961 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e4011b50 con 0x7f25f0108710 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 -- 172.21.15.96:0/504454961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 msgr2=0x7f25f010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:40.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 --2- 172.21.15.96:0/504454961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f25e40098a0 tx=0x7f25e4009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 -- 172.21.15.96:0/504454961 shutdown_connections 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 --2- 172.21.15.96:0/504454961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f010ab00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 -- 172.21.15.96:0/504454961 >> 172.21.15.96:0/504454961 conn(0x7f25f01020e0 msgr2=0x7f25f0104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 -- 172.21.15.96:0/504454961 shutdown_connections 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.098+0000 7f25f51e5700 1 -- 172.21.15.96:0/504454961 wait complete. 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25f51e5700 1 Processor -- start 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25f51e5700 1 -- start start 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25f51e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25f51e5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25f010c060 con 0x7f25f0108710 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25eed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25eed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50160/0 (socket says 172.21.15.96:50160) 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25eed9d700 1 -- 172.21.15.96:0/3877616721 learned_addr learned my addr 172.21.15.96:0/3877616721 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25eed9d700 1 -- 172.21.15.96:0/3877616721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f01084d0 con 0x7f25f0108710 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.099+0000 7f25eed9d700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25e4011670 tx=0x7f25e4011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:40.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e40039c0 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f01066c0 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25e4011e50 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25e401a5c0 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f0106c00 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f25e401a8c0 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.100+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f0063900 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.101+0000 7f25d7fff700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 0x7f25d803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.101+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f25e404d260 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.101+0000 7f25ee59c700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 0x7f25d803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.101+0000 7f25ee59c700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 0x7f25d803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006fd0 tx=0x7f25e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.106+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f25e4019c70 con 0x7f25f0108710 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.300+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f25f01033e0 con 0x7f25d8038ed0 2024-03-18T15:30:40.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.528+0000 7f25d7fff700 1 -- 172.21.15.96:0/3877616721 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25f01033e0 con 0x7f25d8038ed0 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 msgr2=0x7f25d803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 0x7f25d803b390 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f25e0006fd0 tx=0x7f25e0006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 msgr2=0x7f25f0107f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25e4011670 tx=0x7f25e4011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 shutdown_connections 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f25d8038ed0 0x7f25d803b390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 --2- 172.21.15.96:0/3877616721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25f0108710 0x7f25f0107f90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 >> 172.21.15.96:0/3877616721 conn(0x7f25f01020e0 msgr2=0x7f25f0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 shutdown_connections 2024-03-18T15:30:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:39.530+0000 7f25f51e5700 1 -- 172.21.15.96:0/3877616721 wait complete. 2024-03-18T15:30:40.170 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-mon[31930]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:40.170 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:40.170 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:39 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:41.172 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:40 smithi096 ceph-mon[31930]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:41.172 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:40 smithi096 ceph-mon[31930]: Generating ssh key... 2024-03-18T15:30:41.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:30:41.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7eff05197700 1 Processor -- start 2024-03-18T15:30:41.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7eff05197700 1 -- start start 2024-03-18T15:30:41.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7eff05197700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7eff05197700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff00107590 con 0x7eff00106ba0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7efefed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7efefed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50176/0 (socket says 172.21.15.96:50176) 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7efefed9d700 1 -- 172.21.15.96:0/2232380550 learned_addr learned my addr 172.21.15.96:0/2232380550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7efefed9d700 1 -- 172.21.15.96:0/2232380550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff00107da0 con 0x7eff00106ba0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.462+0000 7efefed9d700 1 --2- 172.21.15.96:0/2232380550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efef00098a0 tx=0x7efef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb211365e4a3b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7efefdd9b700 1 -- 172.21.15.96:0/2232380550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efef0004160 con 0x7eff00106ba0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7efefdd9b700 1 -- 172.21.15.96:0/2232380550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efef0004300 con 0x7eff00106ba0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7efefdd9b700 1 -- 172.21.15.96:0/2232380550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efef0011b50 con 0x7eff00106ba0 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 -- 172.21.15.96:0/2232380550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 msgr2=0x7eff00106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 --2- 172.21.15.96:0/2232380550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efef00098a0 tx=0x7efef0009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 -- 172.21.15.96:0/2232380550 shutdown_connections 2024-03-18T15:30:41.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 --2- 172.21.15.96:0/2232380550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff00106fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 -- 172.21.15.96:0/2232380550 >> 172.21.15.96:0/2232380550 conn(0x7eff00102180 msgr2=0x7eff001045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 -- 172.21.15.96:0/2232380550 shutdown_connections 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.463+0000 7eff05197700 1 -- 172.21.15.96:0/2232380550 wait complete. 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7eff05197700 1 Processor -- start 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7eff05197700 1 -- start start 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7eff05197700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7eff05197700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff0010b150 con 0x7eff00106ba0 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7efefed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7efefed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50186/0 (socket says 172.21.15.96:50186) 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7efefed9d700 1 -- 172.21.15.96:0/37403977 learned_addr learned my addr 172.21.15.96:0/37403977 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.464+0000 7efefed9d700 1 -- 172.21.15.96:0/37403977 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0019ec70 con 0x7eff00106ba0 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7efefed9d700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efef0000c00 tx=0x7efef0011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efef0003b20 con 0x7eff00106ba0 2024-03-18T15:30:41.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0019ee40 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efef0003cc0 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efef0024910 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0019f1e0 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.465+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7efef0022020 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.466+0000 7efee7fff700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 0x7efee803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.466+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efef0011e00 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.466+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff00063900 con 0x7eff00106ba0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.466+0000 7efefe59c700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 0x7efee803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.466+0000 7efefe59c700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 0x7efee803b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efef4006fd0 tx=0x7efef4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:41.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.471+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efef00172b0 con 0x7eff00106ba0 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.665+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7eff0002dd20 con 0x7efee8038f20 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.665+0000 7efee7fff700 1 -- 172.21.15.96:0/37403977 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7eff0002dd20 con 0x7efee8038f20 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 msgr2=0x7efee803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 0x7efee803b3e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efef4006fd0 tx=0x7efef4006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 msgr2=0x7eff0007ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efef0000c00 tx=0x7efef0011c10 comp rx=0 tx=0).stop 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 shutdown_connections 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efee8038f20 0x7efee803b3e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 --2- 172.21.15.96:0/37403977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff00106ba0 0x7eff0007ba50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.667+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 >> 172.21.15.96:0/37403977 conn(0x7eff00102180 msgr2=0x7eff00102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:41.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.668+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 shutdown_connections 2024-03-18T15:30:41.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:40.668+0000 7eff05197700 1 -- 172.21.15.96:0/37403977 wait complete. 2024-03-18T15:30:41.264 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:30:41.264 INFO:teuthology.orchestra.run.smithi096.stdout:Adding key to root@localhost authorized_keys... 2024-03-18T15:30:41.264 INFO:teuthology.orchestra.run.smithi096.stdout:Adding host smithi096... 2024-03-18T15:30:42.095 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:41 smithi096 ceph-mon[31930]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:42 smithi096 ceph-mon[31930]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:44.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:43 smithi096 ceph-mon[31930]: Deploying cephadm binary to smithi096 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Added host 'smithi096' with addr '172.21.15.96' 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0cbaf5700 1 Processor -- start 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0cbaf5700 1 -- start start 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0cbaf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0cbaf5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4109c40 con 0x7fe0c4109250 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0c9891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0c9891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50196/0 (socket says 172.21.15.96:50196) 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0c9891700 1 -- 172.21.15.96:0/3862288236 learned_addr learned my addr 172.21.15.96:0/3862288236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.636+0000 7fe0c9891700 1 -- 172.21.15.96:0/3862288236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4109dc0 con 0x7fe0c4109250 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0c9891700 1 --2- 172.21.15.96:0/3862288236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00098a0 tx=0x7fe0c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d417010914387fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0c888f700 1 -- 172.21.15.96:0/3862288236 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c0004160 con 0x7fe0c4109250 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0c888f700 1 -- 172.21.15.96:0/3862288236 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0c0004300 con 0x7fe0c4109250 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0c888f700 1 -- 172.21.15.96:0/3862288236 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c0011b50 con 0x7fe0c4109250 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/3862288236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 msgr2=0x7fe0c4109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/3862288236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00098a0 tx=0x7fe0c0009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/3862288236 shutdown_connections 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/3862288236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c4109670 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:46.617 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.637+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/3862288236 >> 172.21.15.96:0/3862288236 conn(0x7fe0c41018d0 msgr2=0x7fe0c4103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/3862288236 shutdown_connections 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/3862288236 wait complete. 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 Processor -- start 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 -- start start 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0cbaf5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4106ac0 con 0x7fe0c4109250 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0c9891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0c9891700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50200/0 (socket says 172.21.15.96:50200) 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.638+0000 7fe0c9891700 1 -- 172.21.15.96:0/1944766915 learned_addr learned my addr 172.21.15.96:0/1944766915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0c9891700 1 -- 172.21.15.96:0/1944766915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c419c130 con 0x7fe0c4109250 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0c9891700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0011670 tx=0x7fe0c0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:46.618 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c00039c0 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0c0011e50 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c419c420 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c001a5c0 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.639+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c419f0b0 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fe0c001a870 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c4063900 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0baffd700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 0x7fe0b003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe0c004d1d0 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0c9090700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 0x7fe0b003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.640+0000 7fe0c9090700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 0x7fe0b003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.645+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe0c0019c70 con 0x7fe0c4109250 2024-03-18T15:30:46.619 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:41.842+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}) -- 0x7fe0c419f280 con 0x7fe0b0038f20 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.058+0000 7fe0baffd700 1 -- 172.21.15.96:0/1944766915 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe0c419f280 con 0x7fe0b0038f20 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.060+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 msgr2=0x7fe0b003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.060+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 0x7fe0b003b3e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006fd0 tx=0x7fe0b4006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.060+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 msgr2=0x7fe0c419bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.060+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0011670 tx=0x7fe0c0011b60 comp rx=0 tx=0).stop 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 shutdown_connections 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe0b0038f20 0x7fe0b003b3e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 --2- 172.21.15.96:0/1944766915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4109250 0x7fe0c419bbf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 >> 172.21.15.96:0/1944766915 conn(0x7fe0c41018d0 msgr2=0x7fe0c41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 shutdown_connections 2024-03-18T15:30:46.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:46.061+0000 7fe0cbaf5700 1 -- 172.21.15.96:0/1944766915 wait complete. 2024-03-18T15:30:46.621 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mon service... 2024-03-18T15:30:47.223 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:47 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:47.223 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:47 smithi096 ceph-mon[31930]: Added host smithi096 2024-03-18T15:30:47.223 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:47 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:30:48.489 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:48 smithi096 ceph-mon[31930]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:48.489 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:48 smithi096 ceph-mon[31930]: Saving service mon spec with placement count:5 2024-03-18T15:30:48.489 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:48 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:48.490 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:48 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:48.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-18T15:30:48.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f9766327700 1 Processor -- start 2024-03-18T15:30:48.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f9766327700 1 -- start start 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f9766327700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f9766327700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f976007be70 con 0x7f9760108770 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f975ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f975ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35816/0 (socket says 172.21.15.96:35816) 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f975ffff700 1 -- 172.21.15.96:0/1360953023 learned_addr learned my addr 172.21.15.96:0/1360953023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f975ffff700 1 -- 172.21.15.96:0/1360953023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976010b0a0 con 0x7f9760108770 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.070+0000 7f975ffff700 1 --2- 172.21.15.96:0/1360953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97500098a0 tx=0x7f9750009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9506765c6271f0a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f975effd700 1 -- 172.21.15.96:0/1360953023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750004160 con 0x7f9760108770 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f975effd700 1 -- 172.21.15.96:0/1360953023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9750004300 con 0x7f9760108770 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f975effd700 1 -- 172.21.15.96:0/1360953023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750011b50 con 0x7f9760108770 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 -- 172.21.15.96:0/1360953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 msgr2=0x7f976010ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 --2- 172.21.15.96:0/1360953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97500098a0 tx=0x7f9750009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:48.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 -- 172.21.15.96:0/1360953023 shutdown_connections 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 --2- 172.21.15.96:0/1360953023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976010ab60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 -- 172.21.15.96:0/1360953023 >> 172.21.15.96:0/1360953023 conn(0x7f9760102180 msgr2=0x7f97601045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 -- 172.21.15.96:0/1360953023 shutdown_connections 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.071+0000 7f9766327700 1 -- 172.21.15.96:0/1360953023 wait complete. 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f9766327700 1 Processor -- start 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f9766327700 1 -- start start 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f9766327700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f9766327700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f976010c0c0 con 0x7f9760108770 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f975ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f975ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35828/0 (socket says 172.21.15.96:35828) 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.072+0000 7f975ffff700 1 -- 172.21.15.96:0/4085775618 learned_addr learned my addr 172.21.15.96:0/4085775618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f975ffff700 1 -- 172.21.15.96:0/4085775618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976019c4a0 con 0x7f9760108770 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f975ffff700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9750011b60 tx=0x7f9750011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750003b20 con 0x7f9760108770 2024-03-18T15:30:48.749 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9750003cc0 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976019c730 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9750024840 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.073+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976019f3d0 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f9750022020 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9760063900 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f975d7fa700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 0x7f974803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9750011e00 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f975f7fe700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 0x7f974803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.074+0000 7f975f7fe700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 0x7f974803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9754006fd0 tx=0x7f9754006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.079+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9750016260 con 0x7f9760108770 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.295+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f976002dbe0 con 0x7f9748038f20 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.302+0000 7f975d7fa700 1 -- 172.21.15.96:0/4085775618 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f976002dbe0 con 0x7f9748038f20 2024-03-18T15:30:48.750 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 msgr2=0x7f974803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 0x7f974803b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9754006fd0 tx=0x7f9754006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 msgr2=0x7f976019bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9750011b60 tx=0x7f9750011c40 comp rx=0 tx=0).stop 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 shutdown_connections 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f9748038f20 0x7f974803b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 --2- 172.21.15.96:0/4085775618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760108770 0x7f976019bf60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 >> 172.21.15.96:0/4085775618 conn(0x7f9760102180 msgr2=0x7f9760102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 shutdown_connections 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:47.304+0000 7f9766327700 1 -- 172.21.15.96:0/4085775618 wait complete. 2024-03-18T15:30:48.751 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mgr service... 2024-03-18T15:30:49.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:49 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:49.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:49 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:50.247 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.119+0000 7fe2cce06700 1 Processor -- start 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.119+0000 7fe2cce06700 1 -- start start 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2cce06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2cce06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8108670 con 0x7fe2c8107c80 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35830/0 (socket says 172.21.15.96:35830) 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c659c700 1 -- 172.21.15.96:0/174500703 learned_addr learned my addr 172.21.15.96:0/174500703 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c659c700 1 -- 172.21.15.96:0/174500703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c8108e80 con 0x7fe2c8107c80 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c659c700 1 --2- 172.21.15.96:0/174500703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0098a0 tx=0x7fe2bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fa2cd22d4226d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c559a700 1 -- 172.21.15.96:0/174500703 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc004160 con 0x7fe2c8107c80 2024-03-18T15:30:50.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c559a700 1 -- 172.21.15.96:0/174500703 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2bc004300 con 0x7fe2c8107c80 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.120+0000 7fe2c559a700 1 -- 172.21.15.96:0/174500703 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc011b20 con 0x7fe2c8107c80 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 -- 172.21.15.96:0/174500703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 msgr2=0x7fe2c81080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 --2- 172.21.15.96:0/174500703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0098a0 tx=0x7fe2bc009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 -- 172.21.15.96:0/174500703 shutdown_connections 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 --2- 172.21.15.96:0/174500703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c81080a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 -- 172.21.15.96:0/174500703 >> 172.21.15.96:0/174500703 conn(0x7fe2c8103240 msgr2=0x7fe2c8105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:50.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 -- 172.21.15.96:0/174500703 shutdown_connections 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.121+0000 7fe2cce06700 1 -- 172.21.15.96:0/174500703 wait complete. 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2cce06700 1 Processor -- start 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2cce06700 1 -- start start 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2cce06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2cce06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c810c230 con 0x7fe2c8107c80 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:50.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2c659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35844/0 (socket says 172.21.15.96:35844) 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.122+0000 7fe2c659c700 1 -- 172.21.15.96:0/2438571175 learned_addr learned my addr 172.21.15.96:0/2438571175 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2c659c700 1 -- 172.21.15.96:0/2438571175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c819c160 con 0x7fe2c8107c80 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2c659c700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0117e0 tx=0x7fe2bc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc003bf0 con 0x7fe2c8107c80 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2bc01a910 con 0x7fe2c8107c80 2024-03-18T15:30:50.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c819c3f0 con 0x7fe2c8107c80 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2bc024970 con 0x7fe2c8107c80 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.123+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c819c850 con 0x7fe2c8107c80 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.124+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7fe2bc022020 con 0x7fe2c8107c80 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.124+0000 7fe2b77fe700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.124+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe2bc04d150 con 0x7fe2c8107c80 2024-03-18T15:30:50.252 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.124+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c8063900 con 0x7fe2c8107c80 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.124+0000 7fe2c5d9b700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.125+0000 7fe2c5d9b700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 0x7fe2b003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8006fd0 tx=0x7fe2b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.134+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2bc00b3e0 con 0x7fe2c8107c80 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.360+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe2c81044b0 con 0x7fe2b0038f70 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.365+0000 7fe2b77fe700 1 -- 172.21.15.96:0/2438571175 <== mgr.14120 v2:172.21.15.96:6800/2602375712 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe2c81044b0 con 0x7fe2b0038f70 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 msgr2=0x7fe2b003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 0x7fe2b003b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8006fd0 tx=0x7fe2b8006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 msgr2=0x7fe2c819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0117e0 tx=0x7fe2bc011ba0 comp rx=0 tx=0).stop 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 shutdown_connections 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 --2- 172.21.15.96:0/2438571175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2c8107c80 0x7fe2c819bc20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 >> 172.21.15.96:0/2438571175 conn(0x7fe2c8103240 msgr2=0x7fe2c8103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 shutdown_connections 2024-03-18T15:30:50.254 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:49.367+0000 7fe2cce06700 1 -- 172.21.15.96:0/2438571175 wait complete. 2024-03-18T15:30:50.860 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:50 smithi096 ceph-mon[31930]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:30:50.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:50 smithi096 ceph-mon[31930]: Saving service mgr spec with placement count:2 2024-03-18T15:30:51.742 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce6f43700 1 Processor -- start 2024-03-18T15:30:51.742 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce6f43700 1 -- start start 2024-03-18T15:30:51.742 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce6f43700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce6f43700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01075b0 con 0x7f3ce0106bc0 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce4cdf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce4cdf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35860/0 (socket says 172.21.15.96:35860) 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.888+0000 7f3ce4cdf700 1 -- 172.21.15.96:0/1827738251 learned_addr learned my addr 172.21.15.96:0/1827738251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3ce4cdf700 1 -- 172.21.15.96:0/1827738251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce0107dc0 con 0x7f3ce0106bc0 2024-03-18T15:30:51.743 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3ce4cdf700 1 --2- 172.21.15.96:0/1827738251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e8ba008e756149 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3cdf7fe700 1 -- 172.21.15.96:0/1827738251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4004160 con 0x7f3ce0106bc0 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3cdf7fe700 1 -- 172.21.15.96:0/1827738251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd4004300 con 0x7f3ce0106bc0 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3cdf7fe700 1 -- 172.21.15.96:0/1827738251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4011b20 con 0x7f3ce0106bc0 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1827738251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 msgr2=0x7f3ce0106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.889+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1827738251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40098a0 tx=0x7f3cd4009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1827738251 shutdown_connections 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1827738251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce0106fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1827738251 >> 172.21.15.96:0/1827738251 conn(0x7f3ce0102180 msgr2=0x7f3ce01045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1827738251 shutdown_connections 2024-03-18T15:30:51.744 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1827738251 wait complete. 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 Processor -- start 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- start start 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.890+0000 7f3ce6f43700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce010b170 con 0x7f3ce0106bc0 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce4cdf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce4cdf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35874/0 (socket says 172.21.15.96:35874) 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce4cdf700 1 -- 172.21.15.96:0/1516792144 learned_addr learned my addr 172.21.15.96:0/1516792144 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:51.745 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce4cdf700 1 -- 172.21.15.96:0/1516792144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce019b320 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce4cdf700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40117e0 tx=0x7f3cd4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4003bf0 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cd401a910 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce019b5b0 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd4024970 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.891+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce019ba10 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.892+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f3cd4022020 con 0x7f3ce0106bc0 2024-03-18T15:30:51.746 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.892+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0063900 con 0x7f3ce0106bc0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.892+0000 7f3cddffb700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 0x7f3cd003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.892+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3cd404d3a0 con 0x7f3ce0106bc0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.892+0000 7f3cdffff700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 0x7f3cd003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.893+0000 7f3cdffff700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 0x7f3cd003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:50.897+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3cd402cc80 con 0x7f3ce0106bc0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.102+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3ce002dd30 con 0x7f3ce0106bc0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.105+0000 7f3cddffb700 1 -- 172.21.15.96:0/1516792144 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3cd402dc40 con 0x7f3ce0106bc0 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 msgr2=0x7f3cd003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:51.747 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 0x7f3cd003b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc006fd0 tx=0x7f3ccc006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 msgr2=0x7f3ce019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40117e0 tx=0x7f3cd4011ba0 comp rx=0 tx=0).stop 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 shutdown_connections 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f3cd0038f70 0x7f3cd003b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 --2- 172.21.15.96:0/1516792144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ce0106bc0 0x7f3ce019ade0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.107+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 >> 172.21.15.96:0/1516792144 conn(0x7f3ce0102180 msgr2=0x7f3ce0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.108+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 shutdown_connections 2024-03-18T15:30:51.748 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:51.108+0000 7f3ce6f43700 1 -- 172.21.15.96:0/1516792144 wait complete. 2024-03-18T15:30:52.292 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:51 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:52.293 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:51 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1516792144' entity='client.admin' 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb1493f700 1 Processor -- start 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb1493f700 1 -- start start 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb1493f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb1493f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb10107a20 con 0x7feb10107030 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35878/0 (socket says 172.21.15.96:35878) 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.323+0000 7feb0e59c700 1 -- 172.21.15.96:0/1573292411 learned_addr learned my addr 172.21.15.96:0/1573292411 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:53.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.324+0000 7feb0e59c700 1 -- 172.21.15.96:0/1573292411 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb10107d50 con 0x7feb10107030 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.324+0000 7feb0e59c700 1 --2- 172.21.15.96:0/1573292411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb000098a0 tx=0x7feb00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb00cdadf77f51e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.324+0000 7feb0d59a700 1 -- 172.21.15.96:0/1573292411 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb00004160 con 0x7feb10107030 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.324+0000 7feb0d59a700 1 -- 172.21.15.96:0/1573292411 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb00004300 con 0x7feb10107030 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.324+0000 7feb0d59a700 1 -- 172.21.15.96:0/1573292411 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb00011b20 con 0x7feb10107030 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 -- 172.21.15.96:0/1573292411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 msgr2=0x7feb10107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 --2- 172.21.15.96:0/1573292411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb000098a0 tx=0x7feb00009bb0 comp rx=0 tx=0).stop 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 -- 172.21.15.96:0/1573292411 shutdown_connections 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 --2- 172.21.15.96:0/1573292411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10107450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 -- 172.21.15.96:0/1573292411 >> 172.21.15.96:0/1573292411 conn(0x7feb101018d0 msgr2=0x7feb10103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 -- 172.21.15.96:0/1573292411 shutdown_connections 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.325+0000 7feb1493f700 1 -- 172.21.15.96:0/1573292411 wait complete. 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb1493f700 1 Processor -- start 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb1493f700 1 -- start start 2024-03-18T15:30:53.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb1493f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb1493f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010b100 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb0e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35890/0 (socket says 172.21.15.96:35890) 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb0e59c700 1 -- 172.21.15.96:0/2128714192 learned_addr learned my addr 172.21.15.96:0/2128714192 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb0e59c700 1 -- 172.21.15.96:0/2128714192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1019e8b0 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.326+0000 7feb0e59c700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb000117e0 tx=0x7feb00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.327+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb00003bf0 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.327+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1019ea80 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.327+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0001a910 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.327+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb00024970 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.327+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1019ee20 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb10063900 con 0x7feb10107030 2024-03-18T15:30:53.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7feb00022020 con 0x7feb10107030 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feaff7fe700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 0x7feaf803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb0004d150 con 0x7feb10107030 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feb0dd9b700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 0x7feaf803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.328+0000 7feb0dd9b700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 0x7feaf803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feb04006fd0 tx=0x7feb04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.337+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb00011e20 con 0x7feb10107030 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.542+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7feb1019f200 con 0x7feb10107030 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.546+0000 7feaff7fe700 1 -- 172.21.15.96:0/2128714192 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7feb0002ddb0 con 0x7feb10107030 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 msgr2=0x7feaf803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 0x7feaf803b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feb04006fd0 tx=0x7feb04006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 msgr2=0x7feb10106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb000117e0 tx=0x7feb00011c10 comp rx=0 tx=0).stop 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 shutdown_connections 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7feaf8038f20 0x7feaf803b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 --2- 172.21.15.96:0/2128714192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb10107030 0x7feb10106610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:53.133 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.548+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 >> 172.21.15.96:0/2128714192 conn(0x7feb101018d0 msgr2=0x7feb101023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:53.133 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.549+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 shutdown_connections 2024-03-18T15:30:53.133 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:52.549+0000 7feb1493f700 1 -- 172.21.15.96:0/2128714192 wait complete. 2024-03-18T15:30:53.133 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling the dashboard module... 2024-03-18T15:30:53.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:53 smithi096 ceph-mon[31930]: from='mgr.14120 172.21.15.96:0/3633554873' entity='mgr.x' 2024-03-18T15:30:53.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:53 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2128714192' entity='client.admin' 2024-03-18T15:30:54.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:54 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/259373700' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-18T15:30:54.784 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: ignoring --setuser ceph since I am not root 2024-03-18T15:30:54.784 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: ignoring --setgroup ceph since I am not root 2024-03-18T15:30:55.256 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e4b974700 1 Processor -- start 2024-03-18T15:30:55.256 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e4b974700 1 -- start start 2024-03-18T15:30:55.256 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e4b974700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:55.256 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e4b974700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44106ca0 con 0x7f0e441062b0 2024-03-18T15:30:55.256 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e49710700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e49710700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46984/0 (socket says 172.21.15.96:46984) 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.533+0000 7f0e49710700 1 -- 172.21.15.96:0/838908003 learned_addr learned my addr 172.21.15.96:0/838908003 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e49710700 1 -- 172.21.15.96:0/838908003 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e441074b0 con 0x7f0e441062b0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e49710700 1 --2- 172.21.15.96:0/838908003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000aa40 tx=0x7f0e4000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8904707260d570b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e3bfff700 1 -- 172.21.15.96:0/838908003 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e4000b040 con 0x7f0e441062b0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e3bfff700 1 -- 172.21.15.96:0/838908003 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e4000c8f0 con 0x7f0e441062b0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e3bfff700 1 -- 172.21.15.96:0/838908003 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e40003c10 con 0x7f0e441062b0 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e4b974700 1 -- 172.21.15.96:0/838908003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 msgr2=0x7f0e441066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:55.257 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e4b974700 1 --2- 172.21.15.96:0/838908003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000aa40 tx=0x7f0e4000ad50 comp rx=0 tx=0).stop 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e4b974700 1 -- 172.21.15.96:0/838908003 shutdown_connections 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e4b974700 1 --2- 172.21.15.96:0/838908003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e441066d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.534+0000 7f0e4b974700 1 -- 172.21.15.96:0/838908003 >> 172.21.15.96:0/838908003 conn(0x7f0e441018d0 msgr2=0x7f0e44103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 -- 172.21.15.96:0/838908003 shutdown_connections 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 -- 172.21.15.96:0/838908003 wait complete. 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 Processor -- start 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 -- start start 2024-03-18T15:30:55.258 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e4b974700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44107c80 con 0x7f0e441062b0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e49710700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e49710700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46986/0 (socket says 172.21.15.96:46986) 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.535+0000 7f0e49710700 1 -- 172.21.15.96:0/259373700 learned_addr learned my addr 172.21.15.96:0/259373700 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e49710700 1 -- 172.21.15.96:0/259373700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4419c1a0 con 0x7f0e441062b0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e49710700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c550 tx=0x7f0e4000c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e40025070 con 0x7f0e441062b0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e40004540 con 0x7f0e441062b0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e4419c430 con 0x7f0e441062b0 2024-03-18T15:30:55.259 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e40010710 con 0x7f0e441062b0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.536+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e4419f120 con 0x7f0e441062b0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.537+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46106+0+0 (secure 0 0 0) 0x7f0e40028070 con 0x7f0e441062b0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.537+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44063900 con 0x7f0e441062b0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.537+0000 7f0e3a7fc700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 0x7f0e3003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.537+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e4004cc80 con 0x7f0e441062b0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.537+0000 7f0e48f0f700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 0x7f0e3003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:55.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.538+0000 7f0e48f0f700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 0x7f0e3003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.542+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0e40053080 con 0x7f0e441062b0 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:53.814+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0e4402ddc0 con 0x7f0e441062b0 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.478+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f0e40014070 con 0x7f0e441062b0 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.481+0000 7f0e3a7fc700 1 -- 172.21.15.96:0/259373700 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f0e4001dd00 con 0x7f0e441062b0 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 msgr2=0x7f0e3003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:55.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 0x7f0e3003b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34006e40 comp rx=0 tx=0).stop 2024-03-18T15:30:55.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 msgr2=0x7f0e4419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:55.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e4000c550 tx=0x7f0e4000c160 comp rx=0 tx=0).stop 2024-03-18T15:30:55.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 shutdown_connections 2024-03-18T15:30:55.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f0e30038f20 0x7f0e3003b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:55.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 --2- 172.21.15.96:0/259373700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e441062b0 0x7f0e4419bc60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:55.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 >> 172.21.15.96:0/259373700 conn(0x7f0e441018d0 msgr2=0x7f0e44192820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:55.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 shutdown_connections 2024-03-18T15:30:55.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:54.482+0000 7f0e4b974700 1 -- 172.21.15.96:0/259373700 wait complete. 2024-03-18T15:30:55.371 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:55.048+0000 7faef9ba8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:30:55.371 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:55.141+0000 7faef9ba8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:30:55.638 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/259373700' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-18T15:30:55.639 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-mon[31930]: mgrmap e9: x(active, since 19s) 2024-03-18T15:30:55.905 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:55.810+0000 7faef9ba8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:30:55.905 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:55.904+0000 7faef9ba8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:30:56.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.051+0000 7faef9ba8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:30:56.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.147+0000 7faef9ba8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:30:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.664+0000 7f04d0618700 1 Processor -- start 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04d0618700 1 -- start start 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04d0618700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04d0618700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8107540 con 0x7f04c8106b50 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04ce3b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04ce3b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46998/0 (socket says 172.21.15.96:46998) 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.665+0000 7f04ce3b4700 1 -- 172.21.15.96:0/1957424178 learned_addr learned my addr 172.21.15.96:0/1957424178 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04ce3b4700 1 -- 172.21.15.96:0/1957424178 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c8107d50 con 0x7f04c8106b50 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04ce3b4700 1 --2- 172.21.15.96:0/1957424178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04c400bb40 tx=0x7f04c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb64c555dc1f6dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04cd3b2700 1 -- 172.21.15.96:0/1957424178 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c400dbe0 con 0x7f04c8106b50 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04cd3b2700 1 -- 172.21.15.96:0/1957424178 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c4004510 con 0x7f04c8106b50 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04cd3b2700 1 -- 172.21.15.96:0/1957424178 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c4010470 con 0x7f04c8106b50 2024-03-18T15:30:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04d0618700 1 -- 172.21.15.96:0/1957424178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 msgr2=0x7f04c8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.666+0000 7f04d0618700 1 --2- 172.21.15.96:0/1957424178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04c400bb40 tx=0x7f04c400be50 comp rx=0 tx=0).stop 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- 172.21.15.96:0/1957424178 shutdown_connections 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 --2- 172.21.15.96:0/1957424178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c8106f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- 172.21.15.96:0/1957424178 >> 172.21.15.96:0/1957424178 conn(0x7f04c8102130 msgr2=0x7f04c8104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- 172.21.15.96:0/1957424178 shutdown_connections 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- 172.21.15.96:0/1957424178 wait complete. 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 Processor -- start 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- start start 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:56.462 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04d0618700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8074b60 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04ce3b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04ce3b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47012/0 (socket says 172.21.15.96:47012) 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.667+0000 7f04ce3b4700 1 -- 172.21.15.96:0/1211913826 learned_addr learned my addr 172.21.15.96:0/1211913826 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04ce3b4700 1 -- 172.21.15.96:0/1211913826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c819e380 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04ce3b4700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b340 tx=0x7f04c40040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c4010770 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c401d070 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c819e610 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04c4021c30 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.668+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c81a1300 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7f04c401f030 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c8063900 con 0x7f04c8106b50 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04bf7fe700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 0x7f04b403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:30:56.463 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f04c404d1e0 con 0x7f04c8106b50 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04cdbb3700 1 -- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 msgr2=0x7f04b403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.669+0000 7f04cdbb3700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 0x7f04b403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.674+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f04c4015070 con 0x7f04c8106b50 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.870+0000 7f04cdbb3700 1 -- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 msgr2=0x7f04b403b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.870+0000 7f04cdbb3700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 0x7f04b403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.935+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f04c8192f90 con 0x7f04c8106b50 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.936+0000 7f04bf7fe700 1 -- 172.21.15.96:0/1211913826 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f04c8192f90 con 0x7f04c8106b50 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 msgr2=0x7f04b403b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 0x7f04b403b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 msgr2=0x7f04c819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:30:56.464 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b340 tx=0x7f04c40040b0 comp rx=0 tx=0).stop 2024-03-18T15:30:56.465 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 shutdown_connections 2024-03-18T15:30:56.465 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7f04b4038f20 0x7f04b403b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:56.465 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 --2- 172.21.15.96:0/1211913826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04c8106b50 0x7f04c819de40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:30:56.465 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 >> 172.21.15.96:0/1211913826 conn(0x7f04c8102130 msgr2=0x7f04c8190050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:30:56.466 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 shutdown_connections 2024-03-18T15:30:56.466 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:55.938+0000 7f04d0618700 1 -- 172.21.15.96:0/1211913826 wait complete. 2024-03-18T15:30:56.466 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-03-18T15:30:56.466 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 9... 2024-03-18T15:30:56.506 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1211913826' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-18T15:30:56.506 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.233+0000 7faef9ba8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:30:56.788 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.505+0000 7faef9ba8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T15:30:56.788 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.617+0000 7faef9ba8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:56.788 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.707+0000 7faef9ba8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:30:57.093 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.803+0000 7faef9ba8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:30:57.093 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:56.981+0000 7faef9ba8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:30:58.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:57.932+0000 7faef9ba8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T15:30:58.186 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:58.063+0000 7faef9ba8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:30:58.682 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:58.394+0000 7faef9ba8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:30:58.682 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:58.484+0000 7faef9ba8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:30:58.682 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:58.681+0000 7faef9ba8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:58.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:58.869+0000 7faef9ba8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:30:59.423 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:59.158+0000 7faef9ba8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:30:59.423 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:59.422+0000 7faef9ba8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:30:59.686 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:30:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:30:59.661+0000 7faef9ba8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:31:00.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:31:00 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:31:00.464+0000 7faef9ba8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:31:01.347 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:31:01.029+0000 7faef9ba8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:31:01.348 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:31:01.128+0000 7faef9ba8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:31:01.348 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:31:01.346+0000 7faef9ba8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: Active manager daemon x restarted 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: Activating manager daemon x 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: osdmap e3: 0 total, 0 up, 0 in 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: mgrmap e10: x(active, starting, since 0.00348481s) 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:31:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: Manager daemon x is now available 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:01.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:02.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-18T15:31:02.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-18T15:31:02.574 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:02 smithi096 ceph-mon[31930]: mgrmap e11: x(active, since 1.00571s) 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd5e3d8700 1 Processor -- start 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd5e3d8700 1 -- start start 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd5e3d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd5e3d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd58107570 con 0x7efd58106b80 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd57fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:03.279 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd57fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47022/0 (socket says 172.21.15.96:47022) 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd57fff700 1 -- 172.21.15.96:0/897423994 learned_addr learned my addr 172.21.15.96:0/897423994 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd57fff700 1 -- 172.21.15.96:0/897423994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd58107d80 con 0x7efd58106b80 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.887+0000 7efd57fff700 1 --2- 172.21.15.96:0/897423994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efd480098a0 tx=0x7efd48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649383328ec87382 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd56ffd700 1 -- 172.21.15.96:0/897423994 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd48004160 con 0x7efd58106b80 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd56ffd700 1 -- 172.21.15.96:0/897423994 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd48004300 con 0x7efd58106b80 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd56ffd700 1 -- 172.21.15.96:0/897423994 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd48011b20 con 0x7efd58106b80 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd5e3d8700 1 -- 172.21.15.96:0/897423994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 msgr2=0x7efd58106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/897423994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efd480098a0 tx=0x7efd48009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd5e3d8700 1 -- 172.21.15.96:0/897423994 shutdown_connections 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/897423994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd58106fa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.888+0000 7efd5e3d8700 1 -- 172.21.15.96:0/897423994 >> 172.21.15.96:0/897423994 conn(0x7efd58102100 msgr2=0x7efd58104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 -- 172.21.15.96:0/897423994 shutdown_connections 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 -- 172.21.15.96:0/897423994 wait complete. 2024-03-18T15:31:03.280 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 Processor -- start 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 -- start start 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.889+0000 7efd5e3d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd5810b130 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd57fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd57fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47030/0 (socket says 172.21.15.96:47030) 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd57fff700 1 -- 172.21.15.96:0/1188522802 learned_addr learned my addr 172.21.15.96:0/1188522802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd57fff700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5819c1d0 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd57fff700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd48000c00 tx=0x7efd48011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd48003b60 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5819c460 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4801a910 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5819c8c0 con 0x7efd58106b80 2024-03-18T15:31:03.281 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.890+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd48024930 con 0x7efd58106b80 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 46119+0+0 (secure 0 0 0) 0x7efd48022020 con 0x7efd58106b80 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd557fa700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd4003bb40 con 0x7efd40038f70 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efd4804e080 con 0x7efd58106b80 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd577fe700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:56.891+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:57.092+0000 7efd577fe700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:57.092+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:57.493+0000 7efd577fe700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:57.493+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:58.294+0000 7efd577fe700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:58.294+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:59.895+0000 7efd577fe700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/2602375712 2024-03-18T15:31:03.282 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:30:59.895+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:01.362+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 10) ==== 45886+0+0 (secure 0 0 0) 0x7efd4802d3f0 con 0x7efd58106b80 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:01.362+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 msgr2=0x7efd4003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:01.362+0000 7efd557fa700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.363+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7efd4804f080 con 0x7efd58106b80 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.363+0000 7efd557fa700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 0x7efd4003ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.363+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd48022e80 con 0x7efd4003c9f0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.364+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 0x7efd4003ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.364+0000 7efd577fe700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 0x7efd4003ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd4c003a10 tx=0x7efd4c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.365+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7efd48022e80 con 0x7efd4003c9f0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7efd44002630 con 0x7efd4003c9f0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd557fa700 1 -- 172.21.15.96:0/1188522802 <== mgr.14152 v2:172.21.15.96:6800/3036369138 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7efd44002630 con 0x7efd4003c9f0 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 msgr2=0x7efd4003ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:03.283 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 0x7efd4003ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efd4c003a10 tx=0x7efd4c0092b0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 msgr2=0x7efd5819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd48000c00 tx=0x7efd48011c10 comp rx=0 tx=0).stop 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 shutdown_connections 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd4003c9f0 0x7efd4003ede0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd58106b80 0x7efd5819bc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 --2- 172.21.15.96:0/1188522802 >> [v2:172.21.15.96:6800/2602375712,v1:172.21.15.96:6801/2602375712] conn(0x7efd40038f70 0x7efd4003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 >> 172.21.15.96:0/1188522802 conn(0x7efd58102100 msgr2=0x7efd58103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 shutdown_connections 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:02.372+0000 7efd5e3d8700 1 -- 172.21.15.96:0/1188522802 wait complete. 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 9 is available 2024-03-18T15:31:03.284 INFO:teuthology.orchestra.run.smithi096.stdout:Generating a dashboard self-signed certificate... 2024-03-18T15:31:03.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: [18/Mar/2024:15:31:02] ENGINE Bus STARTING 2024-03-18T15:31:03.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: [18/Mar/2024:15:31:02] ENGINE Serving on https://172.21.15.96:7150 2024-03-18T15:31:03.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-18T15:31:03.538 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-18T15:31:03.539 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: [18/Mar/2024:15:31:02] ENGINE Serving on http://172.21.15.96:8765 2024-03-18T15:31:03.539 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: [18/Mar/2024:15:31:02] ENGINE Bus STARTED 2024-03-18T15:31:03.539 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.640+0000 7f3be7194700 1 Processor -- start 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.640+0000 7f3be7194700 1 -- start start 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be7194700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be7194700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3be007be70 con 0x7f3be0108750 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be4f30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:04.575 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be4f30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55298/0 (socket says 172.21.15.96:55298) 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be4f30700 1 -- 172.21.15.96:0/754173955 learned_addr learned my addr 172.21.15.96:0/754173955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be4f30700 1 -- 172.21.15.96:0/754173955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be010b080 con 0x7f3be0108750 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.641+0000 7f3be4f30700 1 --2- 172.21.15.96:0/754173955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40098a0 tx=0x7f3bd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe2b74c9b5780bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3bdf7fe700 1 -- 172.21.15.96:0/754173955 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4004160 con 0x7f3be0108750 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3bdf7fe700 1 -- 172.21.15.96:0/754173955 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4004300 con 0x7f3be0108750 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3bdf7fe700 1 -- 172.21.15.96:0/754173955 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4011b20 con 0x7f3be0108750 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3be7194700 1 -- 172.21.15.96:0/754173955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 msgr2=0x7f3be010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3be7194700 1 --2- 172.21.15.96:0/754173955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40098a0 tx=0x7f3bd4009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3be7194700 1 -- 172.21.15.96:0/754173955 shutdown_connections 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3be7194700 1 --2- 172.21.15.96:0/754173955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be010ab40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.642+0000 7f3be7194700 1 -- 172.21.15.96:0/754173955 >> 172.21.15.96:0/754173955 conn(0x7f3be0102120 msgr2=0x7f3be0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 -- 172.21.15.96:0/754173955 shutdown_connections 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 -- 172.21.15.96:0/754173955 wait complete. 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 Processor -- start 2024-03-18T15:31:04.576 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 -- start start 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be7194700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3be010c0e0 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be4f30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be4f30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55306/0 (socket says 172.21.15.96:55306) 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.643+0000 7f3be4f30700 1 -- 172.21.15.96:0/3753085976 learned_addr learned my addr 172.21.15.96:0/3753085976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3be4f30700 1 -- 172.21.15.96:0/3753085976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be01084c0 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3be4f30700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40117e0 tx=0x7f3bd4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4003bf0 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd401a910 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4024970 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be0106700 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.644+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be0106b60 con 0x7f3be0108750 2024-03-18T15:31:04.577 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.645+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) ==== 46013+0+0 (secure 0 0 0) 0x7f3bd4022020 con 0x7f3be0108750 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.645+0000 7f3bddffb700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 0x7f3bd003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.645+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3bd404d100 con 0x7f3be0108750 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.645+0000 7f3bdffff700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 0x7f3bd003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.645+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0063900 con 0x7f3be0108750 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.646+0000 7f3bdffff700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 0x7f3bd003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc006fd0 tx=0x7f3bcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.651+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3bd4011e20 con 0x7f3be0108750 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.858+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f3be0106de0 con 0x7f3bd0038e10 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.929+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f3bd402d910 con 0x7f3be0108750 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.958+0000 7f3bddffb700 1 -- 172.21.15.96:0/3753085976 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f3be0106de0 con 0x7f3bd0038e10 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 msgr2=0x7f3bd003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:04.578 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 0x7f3bd003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc006fd0 tx=0x7f3bcc006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 msgr2=0x7f3be0107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40117e0 tx=0x7f3bd4011c10 comp rx=0 tx=0).stop 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 shutdown_connections 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3bd0038e10 0x7f3bd003b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 --2- 172.21.15.96:0/3753085976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3be0108750 0x7f3be0107f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 >> 172.21.15.96:0/3753085976 conn(0x7f3be0102120 msgr2=0x7f3be0102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 shutdown_connections 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:03.960+0000 7f3be7194700 1 -- 172.21.15.96:0/3753085976 wait complete. 2024-03-18T15:31:04.579 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial admin user... 2024-03-18T15:31:05.185 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:04 smithi096 ceph-mon[31930]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:05.185 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:04 smithi096 ceph-mon[31930]: mgrmap e12: x(active, since 2s) 2024-03-18T15:31:05.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:05.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:05.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AFQsVs1PPXduJO1EiLl1..Jmk1tjMLEEZyn/TPG/6C3UwX.UAiJvm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710775865, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.937+0000 7f08a0973700 1 Processor -- start 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.937+0000 7f08a0973700 1 -- start start 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.937+0000 7f08a0973700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.937+0000 7f08a0973700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1068e0 con 0x7f089c105ef0 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.937+0000 7f089a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:06.118 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f089a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55318/0 (socket says 172.21.15.96:55318) 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f089a59c700 1 -- 172.21.15.96:0/3941366784 learned_addr learned my addr 172.21.15.96:0/3941366784 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f089a59c700 1 -- 172.21.15.96:0/3941366784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c106a60 con 0x7f089c105ef0 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f089a59c700 1 --2- 172.21.15.96:0/3941366784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f088c0098a0 tx=0x7f088c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bb89e7c9c85081 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f0899d9b700 1 -- 172.21.15.96:0/3941366784 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f088c004160 con 0x7f089c105ef0 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f0899d9b700 1 -- 172.21.15.96:0/3941366784 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f088c004300 con 0x7f089c105ef0 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f0899d9b700 1 -- 172.21.15.96:0/3941366784 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f088c011b20 con 0x7f089c105ef0 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f08a0973700 1 -- 172.21.15.96:0/3941366784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 msgr2=0x7f089c106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.938+0000 7f08a0973700 1 --2- 172.21.15.96:0/3941366784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f088c0098a0 tx=0x7f088c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 -- 172.21.15.96:0/3941366784 shutdown_connections 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 --2- 172.21.15.96:0/3941366784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c106310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 -- 172.21.15.96:0/3941366784 >> 172.21.15.96:0/3941366784 conn(0x7f089c1018d0 msgr2=0x7f089c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 -- 172.21.15.96:0/3941366784 shutdown_connections 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 -- 172.21.15.96:0/3941366784 wait complete. 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 Processor -- start 2024-03-18T15:31:06.119 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.939+0000 7f08a0973700 1 -- start start 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f08a0973700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f08a0973700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f089c10c1a0 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f089a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f089a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55324/0 (socket says 172.21.15.96:55324) 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f089a59c700 1 -- 172.21.15.96:0/813986714 learned_addr learned my addr 172.21.15.96:0/813986714 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f089a59c700 1 -- 172.21.15.96:0/813986714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19c180 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f089a59c700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f088c0117e0 tx=0x7f088c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f088c003bf0 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f088c01a910 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19c410 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f088c0249a0 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.940+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c19c870 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.941+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f088c022020 con 0x7f089c105ef0 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.941+0000 7f0893fff700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 0x7f088803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.941+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f088c04d2b0 con 0x7f089c105ef0 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.941+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f089c063900 con 0x7f089c105ef0 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.941+0000 7f0891d9b700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 0x7f088803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.942+0000 7f0891d9b700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 0x7f088803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0884006fd0 tx=0x7f0884006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:04.946+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f088c011e20 con 0x7f089c105ef0 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.184+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f089c19f1c0 con 0x7f0888038f20 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.411+0000 7f0893fff700 1 -- 172.21.15.96:0/813986714 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f089c19f1c0 con 0x7f0888038f20 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 msgr2=0x7f088803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 0x7f088803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0884006fd0 tx=0x7f0884006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 msgr2=0x7f089c19bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:06.121 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f088c0117e0 tx=0x7f088c011c10 comp rx=0 tx=0).stop 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 shutdown_connections 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0888038f20 0x7f088803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 --2- 172.21.15.96:0/813986714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f089c105ef0 0x7f089c19bc40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 >> 172.21.15.96:0/813986714 conn(0x7f089c1018d0 msgr2=0x7f089c102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 shutdown_connections 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:05.413+0000 7f08a0973700 1 -- 172.21.15.96:0/813986714 wait complete. 2024-03-18T15:31:06.122 INFO:teuthology.orchestra.run.smithi096.stdout:Fetching dashboard port number... 2024-03-18T15:31:06.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:06 smithi096 ceph-mon[31930]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:06.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 8443 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f86135ae700 1 Processor -- start 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f86135ae700 1 -- start start 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f86135ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f86135ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f860c0759c0 con 0x7f860c108740 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f861134a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f861134a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55332/0 (socket says 172.21.15.96:55332) 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f861134a700 1 -- 172.21.15.96:0/87444614 learned_addr learned my addr 172.21.15.96:0/87444614 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.515+0000 7f861134a700 1 -- 172.21.15.96:0/87444614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c10b070 con 0x7f860c108740 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f861134a700 1 --2- 172.21.15.96:0/87444614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0098a0 tx=0x7f85fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855ae4d40f71a0d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:07.583 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f8603fff700 1 -- 172.21.15.96:0/87444614 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85fc004160 con 0x7f860c108740 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f8603fff700 1 -- 172.21.15.96:0/87444614 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc004300 con 0x7f860c108740 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f8603fff700 1 -- 172.21.15.96:0/87444614 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85fc011b20 con 0x7f860c108740 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 -- 172.21.15.96:0/87444614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 msgr2=0x7f860c10ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 --2- 172.21.15.96:0/87444614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0098a0 tx=0x7f85fc009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 -- 172.21.15.96:0/87444614 shutdown_connections 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 --2- 172.21.15.96:0/87444614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c10ab30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 -- 172.21.15.96:0/87444614 >> 172.21.15.96:0/87444614 conn(0x7f860c102130 msgr2=0x7f860c104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.516+0000 7f86135ae700 1 -- 172.21.15.96:0/87444614 shutdown_connections 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f86135ae700 1 -- 172.21.15.96:0/87444614 wait complete. 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f86135ae700 1 Processor -- start 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f86135ae700 1 -- start start 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f86135ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f86135ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f860c10c070 con 0x7f860c108740 2024-03-18T15:31:07.584 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f861134a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f861134a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55340/0 (socket says 172.21.15.96:55340) 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.517+0000 7f861134a700 1 -- 172.21.15.96:0/1960268453 learned_addr learned my addr 172.21.15.96:0/1960268453 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f861134a700 1 -- 172.21.15.96:0/1960268453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860c19c0d0 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f861134a700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0117e0 tx=0x7f85fc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85fc003ba0 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc01a910 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860c19c360 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85fc0248f0 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.518+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f860c19c7c0 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.519+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 46119+0+0 (secure 0 0 0) 0x7f85fc022020 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.519+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860c063900 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.519+0000 7f86027fc700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 0x7f85f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.519+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f85fc04d7d0 con 0x7f860c108740 2024-03-18T15:31:07.585 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.519+0000 7f8610b49700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 0x7f85f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.520+0000 7f8610b49700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 0x7f85f803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8608006fd0 tx=0x7f8608006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.524+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f85fc00b3e0 con 0x7f860c108740 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.728+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f860c19f190 con 0x7f860c108740 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.728+0000 7f86027fc700 1 -- 172.21.15.96:0/1960268453 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f85fc0172b0 con 0x7f860c108740 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 msgr2=0x7f85f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 0x7f85f803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8608006fd0 tx=0x7f8608006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 msgr2=0x7f860c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0117e0 tx=0x7f85fc011ba0 comp rx=0 tx=0).stop 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 shutdown_connections 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85f8038f70 0x7f85f803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 --2- 172.21.15.96:0/1960268453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f860c108740 0x7f860c19bb90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 >> 172.21.15.96:0/1960268453 conn(0x7f860c102130 msgr2=0x7f860c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 shutdown_connections 2024-03-18T15:31:07.586 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:06.730+0000 7f86135ae700 1 -- 172.21.15.96:0/1960268453 wait complete. 2024-03-18T15:31:07.590 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-18T15:31:07.590 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-03-18T15:31:07.597 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-18T15:31:07.597 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-03-18T15:31:07.597 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-03-18T15:31:07.597 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-18T15:31:07.599 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph Dashboard is now available at: 2024-03-18T15:31:07.599 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:07.599 INFO:teuthology.orchestra.run.smithi096.stdout: URL: https://smithi096.front.sepia.ceph.com:8443/ 2024-03-18T15:31:07.599 INFO:teuthology.orchestra.run.smithi096.stdout: User: admin 2024-03-18T15:31:07.600 INFO:teuthology.orchestra.run.smithi096.stdout: Password: 9ysbf2s797 2024-03-18T15:31:07.600 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:07.600 INFO:teuthology.orchestra.run.smithi096.stdout:Saving cluster configuration to /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config directory 2024-03-18T15:31:07.600 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling autotune for osd_memory_target 2024-03-18T15:31:07.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:07 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1960268453' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-18T15:31:08.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:08 smithi096 ceph-mon[31930]: mgrmap e13: x(active, since 6s) 2024-03-18T15:31:08.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7fbd9700 1 Processor -- start 2024-03-18T15:31:08.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7fbd9700 1 -- start start 2024-03-18T15:31:08.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7fbd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:08.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7fbd9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7807c8d0 con 0x7f4c7807dea0 2024-03-18T15:31:08.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7d975700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7d975700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55348/0 (socket says 172.21.15.96:55348) 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7d975700 1 -- 172.21.15.96:0/1115213481 learned_addr learned my addr 172.21.15.96:0/1115213481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.983+0000 7f4c7d975700 1 -- 172.21.15.96:0/1115213481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7807ca50 con 0x7f4c7807dea0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7d975700 1 --2- 172.21.15.96:0/1115213481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c680098a0 tx=0x7f4c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea559c138bdcdc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7c973700 1 -- 172.21.15.96:0/1115213481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c68004160 con 0x7f4c7807dea0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7c973700 1 -- 172.21.15.96:0/1115213481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68004300 con 0x7f4c7807dea0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7c973700 1 -- 172.21.15.96:0/1115213481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c68011b20 con 0x7f4c7807dea0 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/1115213481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 msgr2=0x7f4c7807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/1115213481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c680098a0 tx=0x7f4c68009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/1115213481 shutdown_connections 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/1115213481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7807c300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.984+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/1115213481 >> 172.21.15.96:0/1115213481 conn(0x7f4c78103210 msgr2=0x7f4c78105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/1115213481 shutdown_connections 2024-03-18T15:31:08.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/1115213481 wait complete. 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 Processor -- start 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 -- start start 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7fbd9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7810e2b0 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7d975700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7d975700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55356/0 (socket says 172.21.15.96:55356) 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.985+0000 7f4c7d975700 1 -- 172.21.15.96:0/712200492 learned_addr learned my addr 172.21.15.96:0/712200492 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c7d975700 1 -- 172.21.15.96:0/712200492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7819e5f0 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c7d975700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c680117e0 tx=0x7f4c68011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c68003bf0 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6801a910 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7819e880 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c680249a0 con 0x7f4c7807dea0 2024-03-18T15:31:08.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.986+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7819ece0 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f4c68022020 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c78063900 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c6effd700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 0x7f4c6403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4c6804d350 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c7d174700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 0x7f4c6403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.987+0000 7f4c7d174700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 0x7f4c6403b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006fd0 tx=0x7f4c74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:07.992+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4c68011e20 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.183+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4c7802dce0 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.183+0000 7f4c6effd700 1 -- 172.21.15.96:0/712200492 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4c68019c70 con 0x7f4c7807dea0 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 msgr2=0x7f4c6403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 0x7f4c6403b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006fd0 tx=0x7f4c74006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 msgr2=0x7f4c7819e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c680117e0 tx=0x7f4c68011c10 comp rx=0 tx=0).stop 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 shutdown_connections 2024-03-18T15:31:08.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4c64039020 0x7f4c6403b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:08.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 --2- 172.21.15.96:0/712200492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c7807dea0 0x7f4c7819e0b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:08.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 >> 172.21.15.96:0/712200492 conn(0x7f4c78103210 msgr2=0x7f4c78103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:08.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 shutdown_connections 2024-03-18T15:31:08.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:08.185+0000 7f4c7fbd9700 1 -- 172.21.15.96:0/712200492 wait complete. 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.063+0000 7fa5241d3700 1 Processor -- start 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.063+0000 7fa5241d3700 1 -- start start 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa5241d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa5241d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c107940 con 0x7fa51c1093b0 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa521f6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa521f6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55372/0 (socket says 172.21.15.96:55372) 2024-03-18T15:31:09.968 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa521f6f700 1 -- 172.21.15.96:0/3007817131 learned_addr learned my addr 172.21.15.96:0/3007817131 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa521f6f700 1 -- 172.21.15.96:0/3007817131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c107ac0 con 0x7fa51c1093b0 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa521f6f700 1 --2- 172.21.15.96:0/3007817131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5180098a0 tx=0x7fa518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727d5245b95557eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa520f6d700 1 -- 172.21.15.96:0/3007817131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa518004160 con 0x7fa51c1093b0 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa520f6d700 1 -- 172.21.15.96:0/3007817131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa518004300 con 0x7fa51c1093b0 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.064+0000 7fa520f6d700 1 -- 172.21.15.96:0/3007817131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa518011b20 con 0x7fa51c1093b0 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 -- 172.21.15.96:0/3007817131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 msgr2=0x7fa51c1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 --2- 172.21.15.96:0/3007817131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa5180098a0 tx=0x7fa518009bb0 comp rx=0 tx=0).stop 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 -- 172.21.15.96:0/3007817131 shutdown_connections 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 --2- 172.21.15.96:0/3007817131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c1097d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 -- 172.21.15.96:0/3007817131 >> 172.21.15.96:0/3007817131 conn(0x7fa51c103320 msgr2=0x7fa51c105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 -- 172.21.15.96:0/3007817131 shutdown_connections 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 -- 172.21.15.96:0/3007817131 wait complete. 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.065+0000 7fa5241d3700 1 Processor -- start 2024-03-18T15:31:09.969 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa5241d3700 1 -- start start 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa5241d3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa5241d3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c10c280 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa521f6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa521f6f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55388/0 (socket says 172.21.15.96:55388) 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa521f6f700 1 -- 172.21.15.96:0/1802519471 learned_addr learned my addr 172.21.15.96:0/1802519471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa521f6f700 1 -- 172.21.15.96:0/1802519471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c07e0b0 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa521f6f700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5180117e0 tx=0x7fa518011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa518003bf0 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51801a910 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa518024970 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c07c2a0 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.066+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c07c760 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.067+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fa518022020 con 0x7fa51c1093b0 2024-03-18T15:31:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.067+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c10c400 con 0x7fa51c1093b0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.067+0000 7fa512ffd700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 0x7fa50803b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.067+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa51804d430 con 0x7fa51c1093b0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.068+0000 7fa52176e700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 0x7fa50803b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.068+0000 7fa52176e700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 0x7fa50803b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa50c006fd0 tx=0x7fa50c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.072+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa51802d910 con 0x7fa51c1093b0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.374+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa51c063900 con 0x7fa51c1093b0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.376+0000 7fa512ffd700 1 -- 172.21.15.96:0/1802519471 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fa51802d3f0 con 0x7fa51c1093b0 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 msgr2=0x7fa50803b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 0x7fa50803b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa50c006fd0 tx=0x7fa50c006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 msgr2=0x7fa51c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa5180117e0 tx=0x7fa518011ba0 comp rx=0 tx=0).stop 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 shutdown_connections 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa508039070 0x7fa50803b530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:09.971 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 --2- 172.21.15.96:0/1802519471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa51c1093b0 0x7fa51c07db70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 >> 172.21.15.96:0/1802519471 conn(0x7fa51c103320 msgr2=0x7fa51c103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 shutdown_connections 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-03-18T15:31:09.379+0000 7fa5241d3700 1 -- 172.21.15.96:0/1802519471 wait complete. 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:Or, if you are only running a single cluster on this host: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-18T15:31:09.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph telemetry on 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout:For more information see: 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:09.973 INFO:teuthology.orchestra.run.smithi096.stdout:Bootstrap complete. 2024-03-18T15:31:10.003 INFO:tasks.cephadm:Fetching config... 2024-03-18T15:31:10.003 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:31:10.003 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-18T15:31:10.018 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-18T15:31:10.019 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:31:10.019 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-18T15:31:10.073 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-18T15:31:10.073 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:31:10.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-18T15:31:10.140 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-18T15:31:10.140 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:31:10.141 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-18T15:31:10.197 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-18T15:31:10.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T15:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:10.299 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T15:31:10.346 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:10.361 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-18T15:31:10.409 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxKKdjB6E+aQ7uq1OQJF74ECmWg+mnz0hsArH+fs581LBxiTtJTi45sTMBvLBk7+CpLHw7MQ+R6o5A6d0HBOwnQkK5diUfAlxv/82kxQJbbwdP9NkzJc6NYtZMVsSXtWPpqOwE9IrpS5DSjZWGH9dL9NrAbRZi1G7LqVqXc14jryuEDfxhrjtzKVzpUKuo/LnnD5WEME+u0oO73VF2RF3ALzdi0q5g+R7QEB6UCL7OQ2iadJDJQythPe0vjR4QhvZDvk20nTrU7YGDgZMCEMHf9Yfc2SJ1dJcNDWj603+zOdCYLV6uFfB3WnKU3V0iEmvbALevI8EeAium2mtalA7zIviZX4gZ3zqSEdExofrNPW0bolZ37X1I1/bTImdl8Htr+XEJNLEUJRGF8xHOSi3Sd14sM9/4Ribr3wk7d9Exy6l14ANILLQrLIL3Y3oqDLKVYyE7sQ9nAEEm/qTHy9eiJ1jkAJyzzTCzZv7U9xxVJ13et8FVxyFtLirT0QVaHhs= ceph-419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:10.424 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-18T15:31:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:10 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1802519471' entity='client.admin' 2024-03-18T15:31:10.661 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 -- 172.21.15.96:0/584638841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 msgr2=0x7f4228100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 --2- 172.21.15.96:0/584638841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f4228100ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f421c009910 tx=0x7f421c009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 -- 172.21.15.96:0/584638841 shutdown_connections 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 --2- 172.21.15.96:0/584638841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f4228100ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 -- 172.21.15.96:0/584638841 >> 172.21.15.96:0/584638841 conn(0x7f42280fc170 msgr2=0x7f42280fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 -- 172.21.15.96:0/584638841 shutdown_connections 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.937+0000 7f422ef35700 1 -- 172.21.15.96:0/584638841 wait complete. 2024-03-18T15:31:12.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ef35700 1 Processor -- start 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ef35700 1 -- start start 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ef35700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ef35700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4228106c60 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ccd1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ccd1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55406/0 (socket says 172.21.15.96:55406) 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ccd1700 1 -- 172.21.15.96:0/2884033693 learned_addr learned my addr 172.21.15.96:0/2884033693 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ccd1700 1 -- 172.21.15.96:0/2884033693 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4228197910 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ccd1700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f421c027070 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c00be40 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f421c00f460 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.938+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4228197ba0 con 0x7f4228100bd0 2024-03-18T15:31:12.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.939+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422819a890 con 0x7f4228100bd0 2024-03-18T15:31:12.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f421c02a070 con 0x7f4228100bd0 2024-03-18T15:31:12.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42281073e0 con 0x7f4228100bd0 2024-03-18T15:31:12.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f4225ffb700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 0x7f421803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:12.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f421c01e030 con 0x7f4228100bd0 2024-03-18T15:31:12.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f4227fff700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 0x7f421803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:12.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.940+0000 7f4227fff700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 0x7f421803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4214006fd0 tx=0x7f4214006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:12.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:12.945+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f421c013070 con 0x7f4228100bd0 2024-03-18T15:31:13.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.146+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f422802dc80 con 0x7f4228100bd0 2024-03-18T15:31:13.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.149+0000 7f4225ffb700 1 -- 172.21.15.96:0/2884033693 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f421c0310c0 con 0x7f4228100bd0 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 msgr2=0x7f421803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 0x7f421803b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4214006fd0 tx=0x7f4214006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 msgr2=0x7f42281973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f421c006010 tx=0x7f421c005050 comp rx=0 tx=0).stop 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 shutdown_connections 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4218038fc0 0x7f421803b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 --2- 172.21.15.96:0/2884033693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4228100bd0 0x7f42281973d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:13.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 >> 172.21.15.96:0/2884033693 conn(0x7f42280fc170 msgr2=0x7f422806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:13.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 shutdown_connections 2024-03-18T15:31:13.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:13.151+0000 7f422ef35700 1 -- 172.21.15.96:0/2884033693 wait complete. 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2884033693' entity='client.admin' 2024-03-18T15:31:13.872 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:13.873 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:13.975 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-18T15:31:13.975 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-18T15:31:14.584 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 -- 172.21.15.96:0/3243090582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 msgr2=0x7ff988101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/3243090582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988101020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff978009910 tx=0x7ff978009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 -- 172.21.15.96:0/3243090582 shutdown_connections 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/3243090582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988101020 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 -- 172.21.15.96:0/3243090582 >> 172.21.15.96:0/3243090582 conn(0x7ff9880fc1a0 msgr2=0x7ff9880fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 -- 172.21.15.96:0/3243090582 shutdown_connections 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.036+0000 7ff98d9e9700 1 -- 172.21.15.96:0/3243090582 wait complete. 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff98d9e9700 1 Processor -- start 2024-03-18T15:31:16.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff98d9e9700 1 -- start start 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff98d9e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff98d9e9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff988106c90 con 0x7ff988100c00 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff986ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff986ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46228/0 (socket says 172.21.15.96:46228) 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff986ffd700 1 -- 172.21.15.96:0/160943701 learned_addr learned my addr 172.21.15.96:0/160943701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:16.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff986ffd700 1 -- 172.21.15.96:0/160943701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff988199b70 con 0x7ff988100c00 2024-03-18T15:31:16.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.037+0000 7ff986ffd700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff978005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:16.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.038+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff978027070 con 0x7ff988100c00 2024-03-18T15:31:16.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.038+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff988199e00 con 0x7ff988100c00 2024-03-18T15:31:16.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.038+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff97800be40 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.038+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff97800f460 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.038+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98819caf0 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7ff97802a070 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff98c9e7700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 0x7ff97003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff988107410 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff97801e030 con 0x7ff988100c00 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff9867fc700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 0x7ff97003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:16.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.039+0000 7ff9867fc700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 0x7ff97003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:16.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.044+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff978013070 con 0x7ff988100c00 2024-03-18T15:31:16.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.243+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7ff98802dc10 con 0x7ff970038fc0 2024-03-18T15:31:16.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.246+0000 7ff98c9e7700 1 -- 172.21.15.96:0/160943701 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff98802dc10 con 0x7ff970038fc0 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 msgr2=0x7ff97003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 0x7ff97003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 msgr2=0x7ff988199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff978006010 tx=0x7ff978005050 comp rx=0 tx=0).stop 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 shutdown_connections 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff970038fc0 0x7ff97003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 --2- 172.21.15.96:0/160943701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff988100c00 0x7ff988199630 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 >> 172.21.15.96:0/160943701 conn(0x7ff9880fc1a0 msgr2=0x7ff988190740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 shutdown_connections 2024-03-18T15:31:16.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:16.249+0000 7ff98d9e9700 1 -- 172.21.15.96:0/160943701 wait complete. 2024-03-18T15:31:17.005 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-03-18T15:31:17.005 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:31:17.005 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-03-18T15:31:17.023 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:31:17.024 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:17.081 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-03-18T15:31:17.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch host add smithi137 2024-03-18T15:31:17.314 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:17.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:17.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:17.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:17.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:17.553 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:17.554 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-18T15:31:17.554 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: Updating smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:17.554 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:17.554 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:17.554 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:18.409 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:18 smithi096 ceph-mon[31930]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:18.409 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:18 smithi096 ceph-mon[31930]: Updating smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T15:31:18.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 -- 172.21.15.96:0/3454404103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 msgr2=0x7f75741011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 --2- 172.21.15.96:0/3454404103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f75741011d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7564009910 tx=0x7f7564009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 -- 172.21.15.96:0/3454404103 shutdown_connections 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 --2- 172.21.15.96:0/3454404103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f75741011d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 -- 172.21.15.96:0/3454404103 >> 172.21.15.96:0/3454404103 conn(0x7f75740fc680 msgr2=0x7f75740feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 -- 172.21.15.96:0/3454404103 shutdown_connections 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.721+0000 7f7579b2b700 1 -- 172.21.15.96:0/3454404103 wait complete. 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f7579b2b700 1 Processor -- start 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f7579b2b700 1 -- start start 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f7579b2b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f7579b2b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f757410b1c0 con 0x7f7574100df0 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f75737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f75737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46242/0 (socket says 172.21.15.96:46242) 2024-03-18T15:31:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f75737fe700 1 -- 172.21.15.96:0/465442296 learned_addr learned my addr 172.21.15.96:0/465442296 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f75737fe700 1 -- 172.21.15.96:0/465442296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574198b80 con 0x7f7574100df0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.722+0000 7f75737fe700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7564006010 tx=0x7f7564005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.723+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7564026070 con 0x7f7574100df0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.723+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756400bbd0 con 0x7f7574100df0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.723+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7574198e70 con 0x7f7574100df0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.723+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7564003910 con 0x7f7574100df0 2024-03-18T15:31:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.723+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7574199330 con 0x7f7574100df0 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7564029070 con 0x7f7574100df0 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75741022d0 con 0x7f7574100df0 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7570ff9700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 0x7f755c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f756401e070 con 0x7f7574100df0 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7572ffd700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 0x7f755c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:18.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.724+0000 7f7572ffd700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 0x7f755c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7568006fd0 tx=0x7f7568006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:18.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.729+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7564013070 con 0x7f7574100df0 2024-03-18T15:31:18.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:18.928+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) -- 0x7f7574063900 con 0x7f755c038fc0 2024-03-18T15:31:19.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:19 smithi096 ceph-mon[31930]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:20 smithi096 ceph-mon[31930]: Deploying cephadm binary to smithi137 2024-03-18T15:31:22.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:22 smithi096 ceph-mon[31930]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:23.177 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-03-18T15:31:23.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.177+0000 7f7570ff9700 1 -- 172.21.15.96:0/465442296 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7574063900 con 0x7f755c038fc0 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 msgr2=0x7f755c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 0x7f755c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7568006fd0 tx=0x7f7568006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 msgr2=0x7f7574198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7564006010 tx=0x7f7564005030 comp rx=0 tx=0).stop 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 shutdown_connections 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f755c038fc0 0x7f755c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 --2- 172.21.15.96:0/465442296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7574100df0 0x7f7574198640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 >> 172.21.15.96:0/465442296 conn(0x7f75740fc680 msgr2=0x7f75741027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 shutdown_connections 2024-03-18T15:31:23.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:23.180+0000 7f7579b2b700 1 -- 172.21.15.96:0/465442296 wait complete. 2024-03-18T15:31:23.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T15:31:24.034 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:24.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:24.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:24 smithi096 ceph-mon[31930]: Added host smithi137 2024-03-18T15:31:24.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:24.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:24 smithi096 ceph-mon[31930]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:24.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:25.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 -- 172.21.15.96:0/2279258713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 msgr2=0x7f6b40101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:25.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/2279258713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b40101220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800b0a0 tx=0x7f6b3800b3b0 comp rx=0 tx=0).stop 2024-03-18T15:31:25.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 -- 172.21.15.96:0/2279258713 shutdown_connections 2024-03-18T15:31:25.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/2279258713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b40101220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:25.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 -- 172.21.15.96:0/2279258713 >> 172.21.15.96:0/2279258713 conn(0x7f6b400fc6d0 msgr2=0x7f6b400feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:25.484 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.482+0000 7f6b47ce8700 1 -- 172.21.15.96:0/2279258713 shutdown_connections 2024-03-18T15:31:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b47ce8700 1 -- 172.21.15.96:0/2279258713 wait complete. 2024-03-18T15:31:25.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b47ce8700 1 Processor -- start 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b47ce8700 1 -- start start 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b47ce8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b47ce8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4010b210 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.483+0000 7f6b45a84700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b45a84700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44430/0 (socket says 172.21.15.96:44430) 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b45a84700 1 -- 172.21.15.96:0/241627596 learned_addr learned my addr 172.21.15.96:0/241627596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b45a84700 1 -- 172.21.15.96:0/241627596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b401047d0 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b45a84700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b38015040 tx=0x7f6b38003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b38017040 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b40102e80 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b38022070 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b3801d5d0 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.484+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b40103340 con 0x7f6b40100e40 2024-03-18T15:31:25.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.485+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f6b3801d8d0 con 0x7f6b40100e40 2024-03-18T15:31:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.485+0000 7f6b3effd700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 0x7f6b2803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.485+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b40102320 con 0x7f6b40100e40 2024-03-18T15:31:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.485+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6b3800f050 con 0x7f6b40100e40 2024-03-18T15:31:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.485+0000 7f6b45283700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 0x7f6b2803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:25.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.486+0000 7f6b45283700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 0x7f6b2803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b34006fd0 tx=0x7f6b34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:25.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.490+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b38007e20 con 0x7f6b40100e40 2024-03-18T15:31:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.689+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b40103e10 con 0x7f6b28038fc0 2024-03-18T15:31:25.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.690+0000 7f6b3effd700 1 -- 172.21.15.96:0/241627596 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6b40103e10 con 0x7f6b28038fc0 2024-03-18T15:31:25.694 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:25.694 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 msgr2=0x7f6b2803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 0x7f6b2803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b34006fd0 tx=0x7f6b34006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 msgr2=0x7f6b401028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b38015040 tx=0x7f6b38003d50 comp rx=0 tx=0).stop 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 shutdown_connections 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b28038fc0 0x7f6b2803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 --2- 172.21.15.96:0/241627596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b40100e40 0x7f6b401028e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.692+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 >> 172.21.15.96:0/241627596 conn(0x7f6b400fc6d0 msgr2=0x7f6b4006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.693+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 shutdown_connections 2024-03-18T15:31:25.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:25.693+0000 7f6b47ce8700 1 -- 172.21.15.96:0/241627596 wait complete. 2024-03-18T15:31:26.219 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-03-18T15:31:26.220 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:31:26.220 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-03-18T15:31:26.238 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:31:26.238 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:26.295 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-03-18T15:31:26.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch host add smithi193 2024-03-18T15:31:26.529 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:26.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:26 smithi096 ceph-mon[31930]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:26.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:26 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:27 smithi096 ceph-mon[31930]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 -- 172.21.15.96:0/1364109449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 msgr2=0x7f1b701011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 --2- 172.21.15.96:0/1364109449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b701011f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009910 tx=0x7f1b60009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 -- 172.21.15.96:0/1364109449 shutdown_connections 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 --2- 172.21.15.96:0/1364109449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b701011f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 -- 172.21.15.96:0/1364109449 >> 172.21.15.96:0/1364109449 conn(0x7f1b700fc6a0 msgr2=0x7f1b700feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 -- 172.21.15.96:0/1364109449 shutdown_connections 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.079+0000 7f1b74880700 1 -- 172.21.15.96:0/1364109449 wait complete. 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b74880700 1 Processor -- start 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b74880700 1 -- start start 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b74880700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:28.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b74880700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7010b1e0 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b6e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b6e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44446/0 (socket says 172.21.15.96:44446) 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b6e59c700 1 -- 172.21.15.96:0/450378007 learned_addr learned my addr 172.21.15.96:0/450378007 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.080+0000 7f1b6e59c700 1 -- 172.21.15.96:0/450378007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b70196c60 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b6e59c700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005850 tx=0x7f1b60005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b60026070 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b70196f50 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b6000bbd0 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b60003970 con 0x7f1b70100e10 2024-03-18T15:31:28.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.081+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7019aff0 con 0x7f1b70100e10 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b701022f0 con 0x7f1b70100e10 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f1b60029070 con 0x7f1b70100e10 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b67fff700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 0x7f1b5c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1b6001e070 con 0x7f1b70100e10 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b65d9b700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 0x7f1b5c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:28.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.082+0000 7f1b65d9b700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 0x7f1b5c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006fd0 tx=0x7f1b58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:28.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.087+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1b60013070 con 0x7f1b70100e10 2024-03-18T15:31:28.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:28.288+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) -- 0x7f1b70197a80 con 0x7f1b5c038fc0 2024-03-18T15:31:28.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:28 smithi096 ceph-mon[31930]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:29.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:29.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:29.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:29.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:29.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:29.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:31:29.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:29.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: Deploying cephadm binary to smithi193 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:30 smithi096 ceph-mon[31930]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T15:31:32.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.643+0000 7f1b67fff700 1 -- 172.21.15.96:0/450378007 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1b70197a80 con 0x7f1b5c038fc0 2024-03-18T15:31:32.645 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-03-18T15:31:32.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 msgr2=0x7f1b5c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:32.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 0x7f1b5c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b58006fd0 tx=0x7f1b58006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:32.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 msgr2=0x7f1b70196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:32.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005850 tx=0x7f1b60005030 comp rx=0 tx=0).stop 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 shutdown_connections 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1b5c038fc0 0x7f1b5c03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 --2- 172.21.15.96:0/450378007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b70100e10 0x7f1b70196720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.645+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 >> 172.21.15.96:0/450378007 conn(0x7f1b700fc6a0 msgr2=0x7f1b70103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.646+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 shutdown_connections 2024-03-18T15:31:32.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:32.646+0000 7f1b74880700 1 -- 172.21.15.96:0/450378007 wait complete. 2024-03-18T15:31:32.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:32 smithi096 ceph-mon[31930]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:32.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:32.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:32.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:33.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-18T15:31:33.425 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:33.825 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:33.825 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:33 smithi096 ceph-mon[31930]: Added host smithi193 2024-03-18T15:31:33.826 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:33.826 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:33.826 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:33 smithi096 ceph-mon[31930]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:34.839 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:34 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.888+0000 7fa53706a700 1 -- 172.21.15.96:0/1108558986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 msgr2=0x7fa5301032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.888+0000 7fa53706a700 1 --2- 172.21.15.96:0/1108558986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa5301032e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa524009910 tx=0x7fa524009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- 172.21.15.96:0/1108558986 shutdown_connections 2024-03-18T15:31:34.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 --2- 172.21.15.96:0/1108558986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa5301032e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- 172.21.15.96:0/1108558986 >> 172.21.15.96:0/1108558986 conn(0x7fa530077810 msgr2=0x7fa5300feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- 172.21.15.96:0/1108558986 shutdown_connections 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- 172.21.15.96:0/1108558986 wait complete. 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 Processor -- start 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- start start 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:34.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa53706a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa53010c350 con 0x7fa530102f00 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa534e06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa534e06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59438/0 (socket says 172.21.15.96:59438) 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.889+0000 7fa534e06700 1 -- 172.21.15.96:0/547817490 learned_addr learned my addr 172.21.15.96:0/547817490 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa534e06700 1 -- 172.21.15.96:0/547817490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53019db20 con 0x7fa530102f00 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa534e06700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa524005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa524026070 con 0x7fa530102f00 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa52400bc30 con 0x7fa530102f00 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53019ddb0 con 0x7fa530102f00 2024-03-18T15:31:34.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa524003ab0 con 0x7fa530102f00 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.890+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5301976b0 con 0x7fa530102f00 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.891+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fa524029070 con 0x7fa530102f00 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.891+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa530069f00 con 0x7fa530102f00 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.891+0000 7fa52dffb700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 0x7fa52003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.891+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa52401e070 con 0x7fa530102f00 2024-03-18T15:31:34.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.891+0000 7fa52ffff700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 0x7fa52003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:34.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.892+0000 7fa52ffff700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 0x7fa52003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006fd0 tx=0x7fa51c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:34.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:34.896+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa524013070 con 0x7fa530102f00 2024-03-18T15:31:35.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.094+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa530198be0 con 0x7fa520039020 2024-03-18T15:31:35.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.095+0000 7fa52dffb700 1 -- 172.21.15.96:0/547817490 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fa530198be0 con 0x7fa520039020 2024-03-18T15:31:35.096 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:31:35.096 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 msgr2=0x7fa52003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 0x7fa52003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006fd0 tx=0x7fa51c006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 msgr2=0x7fa53019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa524006010 tx=0x7fa524005050 comp rx=0 tx=0).stop 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 shutdown_connections 2024-03-18T15:31:35.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa520039020 0x7fa52003b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 --2- 172.21.15.96:0/547817490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa530102f00 0x7fa53019d5e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 >> 172.21.15.96:0/547817490 conn(0x7fa530077810 msgr2=0x7fa530103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 shutdown_connections 2024-03-18T15:31:35.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:35.097+0000 7fa53706a700 1 -- 172.21.15.96:0/547817490 wait complete. 2024-03-18T15:31:35.695 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-18T15:31:35.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-18T15:31:35.931 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:31:36.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:36 smithi096 ceph-mon[31930]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:31:36.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:36.354 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:36 smithi096 ceph-mon[31930]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 -- 172.21.15.96:0/3113386793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 msgr2=0x7f2e881022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3113386793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e881022e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009910 tx=0x7f2e80009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 -- 172.21.15.96:0/3113386793 shutdown_connections 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3113386793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e881022e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 -- 172.21.15.96:0/3113386793 >> 172.21.15.96:0/3113386793 conn(0x7f2e880fd810 msgr2=0x7f2e880ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 -- 172.21.15.96:0/3113386793 shutdown_connections 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.243+0000 7f2e8f386700 1 -- 172.21.15.96:0/3113386793 wait complete. 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8f386700 1 Processor -- start 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8f386700 1 -- start start 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8f386700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8f386700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8810c2d0 con 0x7f2e88101f00 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8d122700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8d122700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59444/0 (socket says 172.21.15.96:59444) 2024-03-18T15:31:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8d122700 1 -- 172.21.15.96:0/3514590368 learned_addr learned my addr 172.21.15.96:0/3514590368 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8d122700 1 -- 172.21.15.96:0/3514590368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8819d980 con 0x7f2e88101f00 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.244+0000 7f2e8d122700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005850 tx=0x7f2e80005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80026070 con 0x7f2e88101f00 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8819dc70 con 0x7f2e88101f00 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e8000bbd0 con 0x7f2e88101f00 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e80003970 con 0x7f2e88101f00 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e88197600 con 0x7f2e88101f00 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.245+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f2e80029070 con 0x7f2e88101f00 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.246+0000 7f2e7e7fc700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 0x7f2e7403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.246+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e8001e070 con 0x7f2e88101f00 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.246+0000 7f2e8c921700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 0x7f2e7403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.246+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e881033e0 con 0x7f2e88101f00 2024-03-18T15:31:37.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.246+0000 7f2e8c921700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 0x7f2e7403b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006fd0 tx=0x7f2e78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:37.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.251+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2e80030070 con 0x7f2e88101f00 2024-03-18T15:31:37.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:37.451+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f2e88063900 con 0x7f2e88101f00 2024-03-18T15:31:38.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.427+0000 7f2e7e7fc700 1 -- 172.21.15.96:0/3514590368 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f2e80013070 con 0x7f2e88101f00 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 msgr2=0x7f2e7403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 0x7f2e7403b480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e78006fd0 tx=0x7f2e78006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 msgr2=0x7f2e8819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e80005850 tx=0x7f2e80005030 comp rx=0 tx=0).stop 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 shutdown_connections 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2e74038fc0 0x7f2e7403b480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 --2- 172.21.15.96:0/3514590368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e88101f00 0x7f2e8819d440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 >> 172.21.15.96:0/3514590368 conn(0x7f2e880fd810 msgr2=0x7f2e881038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 shutdown_connections 2024-03-18T15:31:38.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:31:38.430+0000 7f2e8f386700 1 -- 172.21.15.96:0/3514590368 wait complete. 2024-03-18T15:31:38.432 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-03-18T15:31:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3514590368' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:39.134 INFO:tasks.cephadm:Adding mon.a on smithi096 2024-03-18T15:31:39.134 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-03-18T15:31:39.135 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-03-18T15:31:39.135 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch apply mon '3;smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c' 2024-03-18T15:31:39.364 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3514590368' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:40.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:40 smithi096 ceph-mon[31930]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:40.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:40 smithi096 ceph-mon[31930]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T15:31:40.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:40 smithi096 ceph-mon[31930]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:40.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 -- 172.21.15.193:0/1973437201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 msgr2=0x7f74901011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:40.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 --2- 172.21.15.193:0/1973437201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f74901011f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7484009910 tx=0x7f7484009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 -- 172.21.15.193:0/1973437201 shutdown_connections 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 --2- 172.21.15.193:0/1973437201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f74901011f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 -- 172.21.15.193:0/1973437201 >> 172.21.15.193:0/1973437201 conn(0x7f74900fc6a0 msgr2=0x7f74900feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 -- 172.21.15.193:0/1973437201 shutdown_connections 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 -- 172.21.15.193:0/1973437201 wait complete. 2024-03-18T15:31:40.694 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.692+0000 7f7495d58700 1 Processor -- start 2024-03-18T15:31:40.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f7495d58700 1 -- start start 2024-03-18T15:31:40.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f7495d58700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:40.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f7495d58700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f749010b1e0 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f748f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f748f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.193:43522/0 (socket says 172.21.15.193:43522) 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f748f7fe700 1 -- 172.21.15.193:0/3874129863 learned_addr learned my addr 172.21.15.193:0/3874129863 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.693+0000 7f748f7fe700 1 -- 172.21.15.193:0/3874129863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749019d970 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f748f7fe700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7484005850 tx=0x7f7484005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7484026070 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749019dc60 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748400bbd0 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7484003970 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.694+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901975f0 con 0x7f7490100e10 2024-03-18T15:31:40.697 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.695+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74901022f0 con 0x7f7490100e10 2024-03-18T15:31:40.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.695+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f7484029070 con 0x7f7490100e10 2024-03-18T15:31:40.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.695+0000 7f748cff9700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 0x7f747803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:40.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.695+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f748401e070 con 0x7f7490100e10 2024-03-18T15:31:40.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.695+0000 7f748effd700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 0x7f747803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:40.698 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.696+0000 7f748effd700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 0x7f747803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7480006fd0 tx=0x7f7480006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:40.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.700+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7484013070 con 0x7f7490100e10 2024-03-18T15:31:40.906 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.905+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}) -- 0x7f7490198b20 con 0x7f7478038fc0 2024-03-18T15:31:40.914 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-03-18T15:31:40.915 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.913+0000 7f748cff9700 1 -- 172.21.15.193:0/3874129863 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7490198b20 con 0x7f7478038fc0 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 msgr2=0x7f747803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 0x7f747803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7480006fd0 tx=0x7f7480006e40 comp rx=0 tx=0).stop 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 msgr2=0x7f749019d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7484005850 tx=0x7f7484005030 comp rx=0 tx=0).stop 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 shutdown_connections 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7478038fc0 0x7f747803b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 --2- 172.21.15.193:0/3874129863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7490100e10 0x7f749019d430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 >> 172.21.15.193:0/3874129863 conn(0x7f74900fc6a0 msgr2=0x7f749006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 shutdown_connections 2024-03-18T15:31:40.917 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:40.915+0000 7f7495d58700 1 -- 172.21.15.193:0/3874129863 wait complete. 2024-03-18T15:31:41.519 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.b.service 2024-03-18T15:31:41.522 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.c.service 2024-03-18T15:31:41.524 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T15:31:41.524 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T15:31:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c;count:3 2024-03-18T15:31:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: Deploying daemon mon.c on smithi193 2024-03-18T15:31:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:41 smithi096 ceph-mon[31930]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:43.311 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:31:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:44 smithi096 ceph-mon[31930]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 -- 172.21.15.193:0/1857064008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 msgr2=0x7fe2e41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/1857064008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e41023f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009910 tx=0x7fe2dc009c20 comp rx=0 tx=0).stop 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 -- 172.21.15.193:0/1857064008 shutdown_connections 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/1857064008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e41023f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 -- 172.21.15.193:0/1857064008 >> 172.21.15.193:0/1857064008 conn(0x7fe2e40fd900 msgr2=0x7fe2e40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 -- 172.21.15.193:0/1857064008 shutdown_connections 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.251+0000 7fe2e99eb700 1 -- 172.21.15.193:0/1857064008 wait complete. 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.252+0000 7fe2e99eb700 1 Processor -- start 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.252+0000 7fe2e99eb700 1 -- start start 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.252+0000 7fe2e99eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e99eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e410c3e0 con 0x7fe2e4102010 2024-03-18T15:31:45.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.193:43534/0 (socket says 172.21.15.193:43534) 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e2ffd700 1 -- 172.21.15.193:0/3270972478 learned_addr learned my addr 172.21.15.193:0/3270972478 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e2ffd700 1 -- 172.21.15.193:0/3270972478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e419db60 con 0x7fe2e4102010 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e2ffd700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc006010 tx=0x7fe2dc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.253+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc026070 con 0x7fe2e4102010 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.254+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2dc00bbd0 con 0x7fe2e4102010 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.254+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2dc003910 con 0x7fe2e4102010 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.254+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e419de50 con 0x7fe2e4102010 2024-03-18T15:31:45.256 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.254+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e41977a0 con 0x7fe2e4102010 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.254+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e41034f0 con 0x7fe2e4102010 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.255+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fe2dc029070 con 0x7fe2e4102010 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.255+0000 7fe2e89e9700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 0x7fe2cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.255+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe2dc01e070 con 0x7fe2e4102010 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.255+0000 7fe2e27fc700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 0x7fe2cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:45.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.255+0000 7fe2e27fc700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 0x7fe2cc03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800ad30 tx=0x7fe2d80093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:45.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.261+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2dc013070 con 0x7fe2e4102010 2024-03-18T15:31:45.551 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.550+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe2e4063900 con 0x7fe2e4102010 2024-03-18T15:31:45.552 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.550+0000 7fe2e89e9700 1 -- 172.21.15.193:0/3270972478 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe2dc030090 con 0x7fe2e4102010 2024-03-18T15:31:45.552 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:31:45.552 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","modified":"2024-03-18T15:29:58.578642Z","created":"2024-03-18T15:29:58.578642Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.552+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 msgr2=0x7fe2cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.552+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 0x7fe2cc03b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800ad30 tx=0x7fe2d80093f0 comp rx=0 tx=0).stop 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.552+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 msgr2=0x7fe2e419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.552+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc006010 tx=0x7fe2dc005030 comp rx=0 tx=0).stop 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 shutdown_connections 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2cc038fc0 0x7fe2cc03b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 --2- 172.21.15.193:0/3270972478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2e4102010 0x7fe2e419d620 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 >> 172.21.15.193:0/3270972478 conn(0x7fe2e40fd900 msgr2=0x7fe2e41039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:45.554 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 shutdown_connections 2024-03-18T15:31:45.555 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:45.553+0000 7fe2e99eb700 1 -- 172.21.15.193:0/3270972478 wait complete. 2024-03-18T15:31:45.555 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-03-18T15:31:45.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 podman[36670]: 2024-03-18 15:31:45.643266953 +0000 UTC m=+0.346127063 container start 8db2e4fe0093d2bc0f0e3da6680d4188f540a75737aae13d4b25678f8b9791cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T15:31:45.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: set uid:gid to 167:167 (ceph:ceph) 2024-03-18T15:31:45.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev), process ceph-mon, pid 2 2024-03-18T15:31:45.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pidfile_write: ignore empty --pid-file 2024-03-18T15:31:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: load: jerasure load: lrc 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: RocksDB version: 7.9.2 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Git sha 0 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Compile date 2024-03-15 00:33:34 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: DB SUMMARY 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: DB Session ID: QYJUP6DDJ8NTXPMT5NR8 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: CURRENT file: CURRENT 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: IDENTITY file: IDENTITY 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.error_if_exists: 0 2024-03-18T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.create_if_missing: 0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.paranoid_checks: 1 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.env: 0x55fcb696cc60 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.fs: PosixFileSystem 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.info_log: 0x55fcb8a61ac0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_file_opening_threads: 16 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.statistics: (nil) 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.use_fsync: 0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_log_file_size: 0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-18T15:31:45.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.keep_log_file_num: 1000 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.recycle_log_file_num: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_fallocate: 1 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_mmap_reads: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_mmap_writes: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.use_direct_reads: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.create_missing_column_families: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.db_log_dir: 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.wal_dir: 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-18T15:31:45.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.advise_random_on_open: 1 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.db_write_buffer_size: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.write_buffer_manager: 0x55fcb8a341e0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.rate_limiter: (nil) 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.wal_recovery_mode: 2 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enable_thread_tracking: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enable_pipelined_write: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.unordered_write: 0 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-18T15:31:45.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.row_cache: None 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.wal_filter: None 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_ingest_behind: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.two_write_queues: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.manual_wal_flush: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.wal_compression: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.atomic_flush: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.log_readahead_size: 0 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-18T15:31:45.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.best_efforts_recovery: 0 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.allow_data_in_errors: 0 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.db_host_id: __hostname__ 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_background_jobs: 2 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_background_compactions: -1 2024-03-18T15:31:45.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_subcompactions: 1 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_total_wal_size: 0 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_open_files: -1 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bytes_per_sync: 0 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-18T15:31:45.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_readahead_size: 0 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_background_flushes: -1 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Compression algorithms supported: 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kZSTD supported: 0 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kXpressCompression supported: 0 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kLZ4HCCompression supported: 1 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kZlibCompression supported: 1 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kSnappyCompression supported: 1 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kLZ4Compression supported: 1 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: kBZip2Compression supported: 0 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-18T15:31:45.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.merge_operator: 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_filter: None 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_filter_factory: None 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.sst_partitioner_factory: None 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fcb8a61bc0) 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-18T15:31:45.928 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x55fcb8a67090 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-03-18T15:31:45.929 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-03-18T15:31:45.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.write_buffer_size: 33554432 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_write_buffer_number: 2 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression: NoCompression 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression: Disabled 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.prefix_extractor: nullptr 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.num_levels: 7 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-18T15:31:45.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.level: 32767 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.strategy: 0 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.enabled: false 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-18T15:31:45.932 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.target_file_size_base: 67108864 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-18T15:31:45.933 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.arena_block_size: 1048576 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.disable_auto_compactions: 0 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.table_properties_collectors: 2024-03-18T15:31:45.934 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.inplace_update_support: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.bloom_locality: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.max_successive_merges: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.paranoid_file_checks: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.force_consistency_checks: 1 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.report_bg_io_stats: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.ttl: 2592000 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-18T15:31:45.935 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enable_blob_files: false 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.min_blob_size: 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_file_size: 268435456 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.blob_file_starting_level: 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e37b5a10-e17c-4d2b-b8ea-b0f220b9cd93 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775905673623, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-18T15:31:45.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775905674068, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710775905, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e37b5a10-e17c-4d2b-b8ea-b0f220b9cd93", "db_session_id": "QYJUP6DDJ8NTXPMT5NR8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710775905674148, "job": 1, "event": "recovery_finished"} 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fcb8b5a000 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: DB pointer 0x55fcb8b44000 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T15:31:45.937 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-18T15:31:45.938 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-18T15:31:45.939 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x55fcb8a67090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(???) e0 preinit fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).mds e1 new map 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-03-18T15:31:45.940 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:31:45.941 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Added host smithi137 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.942 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:45.943 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Deploying cephadm binary to smithi193 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Added host smithi193 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:31:45.944 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3514590368' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3514590368' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:45.945 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c;count:3 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:31:45.946 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: Deploying daemon mon.c on smithi193 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 ceph-mon[36702]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 bash[36670]: 8db2e4fe0093d2bc0f0e3da6680d4188f540a75737aae13d4b25678f8b9791cd 2024-03-18T15:31:45.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:45 smithi193 systemd[1]: Started Ceph mon.c for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:31:47.380 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T15:31:47.380 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T15:31:47.624 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: mon.a calling monitor election 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: mon.c calling monitor election 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:51.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: monmap epoch 2 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: last_changed 2024-03-18T15:31:45.696346+0000 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: min_mon_release 19 (squid) 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: election_strategy: 1 2024-03-18T15:31:51.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: fsmap 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: mgrmap e13: x(active, since 49s) 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: overall HEALTH_OK 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:31:51.171 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:51.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: mon.a calling monitor election 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: mon.c calling monitor election 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: monmap epoch 2 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:51.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: last_changed 2024-03-18T15:31:45.696346+0000 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: min_mon_release 19 (squid) 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: election_strategy: 1 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: fsmap 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: mgrmap e13: x(active, since 49s) 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: overall HEALTH_OK 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:31:51.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:31:52.051 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 -- 172.21.15.193:0/492419209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70400035c0 msgr2=0x7f7040005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:52.051 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 --2- 172.21.15.193:0/492419209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70400035c0 0x7f7040005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7050101ce0 tx=0x7f704400feb0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.051 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 -- 172.21.15.193:0/492419209 shutdown_connections 2024-03-18T15:31:52.051 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 --2- 172.21.15.193:0/492419209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70400035c0 0x7f7040005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.052 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 -- 172.21.15.193:0/492419209 >> 172.21.15.193:0/492419209 conn(0x7f70500fd8a0 msgr2=0x7f70500ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:52.052 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 -- 172.21.15.193:0/492419209 shutdown_connections 2024-03-18T15:31:52.052 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.051+0000 7f7057186700 1 -- 172.21.15.193:0/492419209 wait complete. 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 Processor -- start 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 -- start start 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 0x7f70501977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7050109e40 con 0x7f705019dbe0 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7057186700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7050109cc0 con 0x7f70400035c0 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:38172/0 (socket says 172.21.15.193:38172) 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 -- 172.21.15.193:0/1058696328 learned_addr learned my addr 172.21.15.193:0/1058696328 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 -- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 msgr2=0x7f705019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 -- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 msgr2=0x7f705019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T15:31:52.053 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:31:52.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.052+0000 7f7054f22700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:31:52.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.053+0000 7f704ffff700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 0x7f70501977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:52.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.054+0000 7f704ffff700 1 -- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 msgr2=0x7f705019d6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:31:52.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.054+0000 7f704ffff700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.054+0000 7f704ffff700 1 -- 172.21.15.193:0/1058696328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705019e160 con 0x7f705019dbe0 2024-03-18T15:31:52.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.055+0000 7f704ffff700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 0x7f70501977c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4d0 tx=0x7f703c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:52.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.055+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f703c011630 con 0x7f705019dbe0 2024-03-18T15:31:52.055 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.055+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c011cb0 con 0x7f705019dbe0 2024-03-18T15:31:52.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.055+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7050197d60 con 0x7f705019dbe0 2024-03-18T15:31:52.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.055+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f703c010e60 con 0x7f705019dbe0 2024-03-18T15:31:52.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:51 smithi193 ceph-mon[36702]: Deploying daemon mon.b on smithi137 2024-03-18T15:31:52.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:51 smithi193 ceph-mon[36702]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:52.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:51 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:52.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.056+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501982f0 con 0x7f705019dbe0 2024-03-18T15:31:52.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.057+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f703c018020 con 0x7f705019dbe0 2024-03-18T15:31:52.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.057+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70501034f0 con 0x7f705019dbe0 2024-03-18T15:31:52.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.057+0000 7f704dffb700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 0x7f703403b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:31:52.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.057+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f703c04ba60 con 0x7f705019dbe0 2024-03-18T15:31:52.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.057+0000 7f7054f22700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 0x7f703403b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:31:52.058 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.058+0000 7f7054f22700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 0x7f703403b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f704400fa60 tx=0x7f7044005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:31:52.063 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.063+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f703c01f020 con 0x7f705019dbe0 2024-03-18T15:31:52.087 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:51 smithi096 ceph-mon[31930]: Deploying daemon mon.b on smithi137 2024-03-18T15:31:52.087 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:51 smithi096 ceph-mon[31930]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:52.088 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:51 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:52.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.344+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7050198660 con 0x7f705019dbe0 2024-03-18T15:31:52.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.344+0000 7f704dffb700 1 -- 172.21.15.193:0/1058696328 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f703c018570 con 0x7f705019dbe0 2024-03-18T15:31:52.345 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:31:52.345 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","modified":"2024-03-18T15:31:45.696346Z","created":"2024-03-18T15:29:58.578642Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-18T15:31:52.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 msgr2=0x7f703403b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:52.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 0x7f703403b220 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f704400fa60 tx=0x7f7044005a90 comp rx=0 tx=0).stop 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 msgr2=0x7f70501977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 0x7f70501977c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f703c00b4d0 tx=0x7f703c00b890 comp rx=0 tx=0).stop 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 shutdown_connections 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7034038d60 0x7f703403b220 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70400035c0 0x7f705019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 --2- 172.21.15.193:0/1058696328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f705019dbe0 0x7f70501977c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 >> 172.21.15.193:0/1058696328 conn(0x7f70500fd8a0 msgr2=0x7f70500fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 shutdown_connections 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:31:52.347+0000 7f7057186700 1 -- 172.21.15.193:0/1058696328 wait complete. 2024-03-18T15:31:52.348 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-03-18T15:31:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:52 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/1058696328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T15:31:53.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:52 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/1058696328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T15:31:53.989 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-18T15:31:53.989 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-18T15:31:54.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:53 smithi193 ceph-mon[36702]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:54.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:53 smithi096 ceph-mon[31930]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:54.226 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:31:59.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:31:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.892 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: mon.a calling monitor election 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: mon.c calling monitor election 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: mon.b calling monitor election 2024-03-18T15:31:59.893 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: monmap epoch 3 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: last_changed 2024-03-18T15:31:54.369297+0000 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:31:59.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: min_mon_release 19 (squid) 2024-03-18T15:31:59.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: election_strategy: 1 2024-03-18T15:31:59.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:31:59.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: fsmap 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: mgrmap e13: x(active, since 58s) 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: overall HEALTH_OK 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.898 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:31:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:31:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: mon.a calling monitor election 2024-03-18T15:31:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: mon.c calling monitor election 2024-03-18T15:31:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: mon.b calling monitor election 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: monmap epoch 3 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: last_changed 2024-03-18T15:31:54.369297+0000 2024-03-18T15:31:59.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: created 2024-03-18T15:29:58.578642+0000 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: min_mon_release 19 (squid) 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: election_strategy: 1 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: fsmap 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: osdmap e4: 0 total, 0 up, 0 in 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: mgrmap e13: x(active, since 58s) 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: overall HEALTH_OK 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:31:59.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:31:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:00.016 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.015+0000 7f1316a94700 1 -- 172.21.15.193:0/2517233803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1310100ef0 msgr2=0x7f12fc005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.015+0000 7f1316a94700 1 --2- 172.21.15.193:0/2517233803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1310100ef0 0x7f12fc005610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1308009c10 tx=0x7f1308009f20 comp rx=0 tx=0).stop 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.016+0000 7f1316a94700 1 -- 172.21.15.193:0/2517233803 shutdown_connections 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.016+0000 7f1316a94700 1 --2- 172.21.15.193:0/2517233803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1310100ef0 0x7f12fc005610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.016+0000 7f1316a94700 1 -- 172.21.15.193:0/2517233803 >> 172.21.15.193:0/2517233803 conn(0x7f13100fc7a0 msgr2=0x7f13100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.016+0000 7f1316a94700 1 -- 172.21.15.193:0/2517233803 shutdown_connections 2024-03-18T15:32:00.017 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.016+0000 7f1316a94700 1 -- 172.21.15.193:0/2517233803 wait complete. 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 Processor -- start 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 -- start start 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 0x7f131019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:00.019 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1310108cd0 con 0x7f131019b5c0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1310108b50 con 0x7f1310100ef0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.017+0000 7f1316a94700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1310108e50 con 0x7f1310196ab0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:40154/0 (socket says 172.21.15.193:40154) 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 -- 172.21.15.193:0/3053644376 learned_addr learned my addr 172.21.15.193:0/3053644376 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1314830700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1314830700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 msgr2=0x7f1310196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 0x7f131019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1314830700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 msgr2=0x7f1310196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1314830700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1314830700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 msgr2=0x7f1310196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 msgr2=0x7f1310196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-18T15:32:00.020 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130ffff700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 msgr2=0x7f1310196f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 msgr2=0x7f1310196570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 -- 172.21.15.193:0/3053644376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131019c210 con 0x7f131019b5c0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f1315031700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 0x7f131019ba50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.018+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8015020 con 0x7f131019b5c0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.019+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f8011d10 con 0x7f131019b5c0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.019+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8010e10 con 0x7f131019b5c0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.019+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f131019c500 con 0x7f131019b5c0 2024-03-18T15:32:00.021 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.019+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f131019c990 con 0x7f131019b5c0 2024-03-18T15:32:00.022 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.020+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f12f8018020 con 0x7f131019b5c0 2024-03-18T15:32:00.022 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.020+0000 7f130dffb700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 0x7f12f003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:00.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.020+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12f804b970 con 0x7f131019b5c0 2024-03-18T15:32:00.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.020+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1310063900 con 0x7f131019b5c0 2024-03-18T15:32:00.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.020+0000 7f1314830700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 0x7f12f003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:00.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.021+0000 7f1314830700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 0x7f12f003b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1308009a50 tx=0x7f130800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:00.026 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.026+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f12f801f020 con 0x7f131019b5c0 2024-03-18T15:32:00.335 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.335+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f13101ab4f0 con 0x7f131019b5c0 2024-03-18T15:32:00.335 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.335+0000 7f130dffb700 1 -- 172.21.15.193:0/3053644376 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7f12f801a960 con 0x7f131019b5c0 2024-03-18T15:32:00.336 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-03-18T15:32:00.336 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","modified":"2024-03-18T15:31:54.369297Z","created":"2024-03-18T15:29:58.578642Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.337+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 msgr2=0x7f12f003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.337+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 0x7f12f003b910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1308009a50 tx=0x7f130800b040 comp rx=0 tx=0).stop 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.337+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 msgr2=0x7f131019ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.337+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 0x7f131019ba50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f800b7e0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 shutdown_connections 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12f0039450 0x7f12f003b910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.338 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f131019b5c0 0x7f131019ba50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310100ef0 0x7f1310196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 --2- 172.21.15.193:0/3053644376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1310196ab0 0x7f1310196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 >> 172.21.15.193:0/3053644376 conn(0x7f13100fc7a0 msgr2=0x7f13100fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 shutdown_connections 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:00.338+0000 7f1316a94700 1 -- 172.21.15.193:0/3053644376 wait complete. 2024-03-18T15:32:00.339 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-03-18T15:32:00.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:00 smithi193 ceph-mon[36702]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:00.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:00.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:00 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/3053644376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:32:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:00 smithi137 ceph-mon[35868]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:00 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/3053644376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:32:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:00 smithi096 ceph-mon[31930]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:00 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/3053644376' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-18T15:32:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:32:01.193 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-18T15:32:01.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-18T15:32:01.440 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:32:02.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:02.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:02.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:02.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:02.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:02.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:02.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 -- 172.21.15.96:0/55850594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 msgr2=0x7f5670100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 --2- 172.21.15.96:0/55850594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f5670100e40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5660009910 tx=0x7f5660009c20 comp rx=0 tx=0).stop 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 -- 172.21.15.96:0/55850594 shutdown_connections 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 --2- 172.21.15.96:0/55850594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f5670100e40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 -- 172.21.15.96:0/55850594 >> 172.21.15.96:0/55850594 conn(0x7f56700fc680 msgr2=0x7f56700feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 -- 172.21.15.96:0/55850594 shutdown_connections 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.931+0000 7f567583f700 1 -- 172.21.15.96:0/55850594 wait complete. 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 Processor -- start 2024-03-18T15:32:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 -- start start 2024-03-18T15:32:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f567019fe50 0x7f5670199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f567019a010 0x7f567019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f567010e4d0 con 0x7f5670100a60 2024-03-18T15:32:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f567010e350 con 0x7f567019a010 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f567583f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f567010e650 con 0x7f567019fe50 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f566effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f566effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53964/0 (socket says 172.21.15.96:53964) 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f566effd700 1 -- 172.21.15.96:0/460565140 learned_addr learned my addr 172.21.15.96:0/460565140 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f566e7fc700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f567019fe50 0x7f5670199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.932+0000 7f566f7fe700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f567019a010 0x7f567019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 -- 172.21.15.96:0/460565140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f567019fe50 msgr2=0x7f5670199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f567019fe50 0x7f5670199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 -- 172.21.15.96:0/460565140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f567019a010 msgr2=0x7f567019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f567019a010 0x7f567019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 -- 172.21.15.96:0/460565140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56701a67e0 con 0x7f5670100a60 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f566effd700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f566000bdc0 tx=0x7f566000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660026070 con 0x7f5670100a60 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56600050d0 con 0x7f5670100a60 2024-03-18T15:32:02.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56701a69b0 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566000fdf0 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.933+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56701a6e90 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f5660029070 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f567483d700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 0x7f565803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5670063900 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f566004cbb0 con 0x7f5670100a60 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f566e7fc700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 0x7f565803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:02.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.934+0000 7f566e7fc700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 0x7f565803b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5664009850 tx=0x7f5664008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:02.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:02.939+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5660030020 con 0x7f5670100a60 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.139+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f56701a7060 con 0x7f5670100a60 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.139+0000 7f567483d700 1 -- 172.21.15.96:0/460565140 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f5660013070 con 0x7f5670100a60 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:# minimal ceph.conf for 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:32:03.140 INFO:teuthology.orchestra.run.smithi096.stdout: mon_host = [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-03-18T15:32:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 msgr2=0x7f565803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 0x7f565803b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5664009850 tx=0x7f5664008040 comp rx=0 tx=0).stop 2024-03-18T15:32:03.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 msgr2=0x7f567019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f566000bdc0 tx=0x7f566000bea0 comp rx=0 tx=0).stop 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 shutdown_connections 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5658039450 0x7f565803b910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f567019a010 0x7f567019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5670100a60 0x7f567019f910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 --2- 172.21.15.96:0/460565140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f567019fe50 0x7f5670199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 >> 172.21.15.96:0/460565140 conn(0x7f56700fc680 msgr2=0x7f56700fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.141+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 shutdown_connections 2024-03-18T15:32:03.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:03.142+0000 7f567583f700 1 -- 172.21.15.96:0/460565140 wait complete. 2024-03-18T15:32:03.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: Reconfiguring daemon mon.a on smithi096 2024-03-18T15:32:03.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:03 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/460565140' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: Reconfiguring daemon mon.a on smithi096 2024-03-18T15:32:03.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:03 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/460565140' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi096:/etc/ceph/ceph.conf 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi193:/etc/ceph/ceph.conf 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Updating smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Reconfiguring mon.a (unknown last config time)... 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.670 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: Reconfiguring daemon mon.a on smithi096 2024-03-18T15:32:03.670 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:03 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/460565140' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:03.688 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-18T15:32:03.688 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:32:03.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T15:32:03.717 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:32:03.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:32:03.783 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:32:03.783 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T15:32:03.818 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:32:03.818 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:32:03.888 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:32:03.888 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-03-18T15:32:03.923 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:32:03.923 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-18T15:32:03.993 INFO:tasks.cephadm:Adding mgr.x on smithi096 2024-03-18T15:32:03.994 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-03-18T15:32:03.994 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-03-18T15:32:03.994 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch apply mgr '3;smithi096=x;smithi137=y;smithi193=z' 2024-03-18T15:32:04.269 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:32:04.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:04 smithi096 ceph-mon[31930]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:04.471 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:04 smithi193 ceph-mon[36702]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:04.603 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:04 smithi137 ceph-mon[35868]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 -- 172.21.15.193:0/2522150096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c103490 msgr2=0x7fca2c1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 --2- 172.21.15.193:0/2522150096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c103490 0x7fca2c1038b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009910 tx=0x7fca1c009c20 comp rx=0 tx=0).stop 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 -- 172.21.15.193:0/2522150096 shutdown_connections 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 --2- 172.21.15.193:0/2522150096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c103490 0x7fca2c1038b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 -- 172.21.15.193:0/2522150096 >> 172.21.15.193:0/2522150096 conn(0x7fca2c0fd3f0 msgr2=0x7fca2c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 -- 172.21.15.193:0/2522150096 shutdown_connections 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.673+0000 7fca32a60700 1 -- 172.21.15.193:0/2522150096 wait complete. 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 Processor -- start 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 -- start start 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c197c60 0x7fca2c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:05.676 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2c1980e0 0x7fca2c19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c107f00 con 0x7fca2c197c60 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c107d80 con 0x7fca2c1980e0 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca32a60700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c108080 con 0x7fca2c103490 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca2bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca2bfff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:47520/0 (socket says 172.21.15.193:47520) 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.674+0000 7fca2bfff700 1 -- 172.21.15.193:0/3533680101 learned_addr learned my addr 172.21.15.193:0/3533680101 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2b7fe700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c197c60 0x7fca2c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca30ffd700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2c1980e0 0x7fca2c19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 -- 172.21.15.193:0/3533680101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2c1980e0 msgr2=0x7fca2c19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2c1980e0 0x7fca2c19f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 -- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c197c60 msgr2=0x7fca2c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c197c60 0x7fca2c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 -- 172.21.15.193:0/3533680101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c19fb30 con 0x7fca2c103490 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca2bfff700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005000 tx=0x7fca1c005030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:05.677 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c026070 con 0x7fca2c103490 2024-03-18T15:32:05.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca1c005dd0 con 0x7fca2c103490 2024-03-18T15:32:05.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c19fdc0 con 0x7fca2c103490 2024-03-18T15:32:05.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c017440 con 0x7fca2c103490 2024-03-18T15:32:05.678 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.675+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c1a0280 con 0x7fca2c103490 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.676+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7fca1c029070 con 0x7fca2c103490 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.676+0000 7fca297fa700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 0x7fca1403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.676+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca1c04dd10 con 0x7fca2c103490 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.676+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c063900 con 0x7fca2c103490 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.676+0000 7fca2b7fe700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 0x7fca1403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:05.679 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.677+0000 7fca2b7fe700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 0x7fca1403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca20009ad0 tx=0x7fca20009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:05.682 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.682+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fca1c013070 con 0x7fca2c103490 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: Reconfiguring mon.b (monmap changed)... 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:05.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:05 smithi137 ceph-mon[35868]: Reconfiguring daemon mon.b on smithi137 2024-03-18T15:32:05.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.889+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi137=y;smithi193=z", "target": ["mon-mgr", ""]}) -- 0x7fca2c062710 con 0x7fca14039450 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: Reconfiguring mon.b (monmap changed)... 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:05.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:05 smithi193 ceph-mon[36702]: Reconfiguring daemon mon.b on smithi137 2024-03-18T15:32:05.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.896+0000 7fca297fa700 1 -- 172.21.15.193:0/3533680101 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fca2c062710 con 0x7fca14039450 2024-03-18T15:32:05.896 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 msgr2=0x7fca1403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 0x7fca1403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca20009ad0 tx=0x7fca20009210 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 msgr2=0x7fca2c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca1c005000 tx=0x7fca1c005030 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 shutdown_connections 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fca14039450 0x7fca1403b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca2c1980e0 0x7fca2c19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2c103490 0x7fca2c197720 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 --2- 172.21.15.193:0/3533680101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca2c197c60 0x7fca2c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 >> 172.21.15.193:0/3533680101 conn(0x7fca2c0fd3f0 msgr2=0x7fca2c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:05.899 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 shutdown_connections 2024-03-18T15:32:05.900 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:32:05.898+0000 7fca32a60700 1 -- 172.21.15.193:0/3533680101 wait complete. 2024-03-18T15:32:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: Reconfiguring mon.b (monmap changed)... 2024-03-18T15:32:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:05.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:05.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:05.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:05 smithi096 ceph-mon[31930]: Reconfiguring daemon mon.b on smithi137 2024-03-18T15:32:06.486 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.y.service 2024-03-18T15:32:06.489 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.z.service 2024-03-18T15:32:06.491 INFO:tasks.cephadm:Deploying OSDs... 2024-03-18T15:32:06.491 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:32:06.491 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T15:32:06.510 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T15:32:06.510 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-18 15:31:13.663254480 +0000 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-18 15:26:19.118686840 +0000 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-18 15:26:19.118686840 +0000 2024-03-18T15:32:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-18 15:26:19.118686840 +0000 2024-03-18T15:32:06.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T15:32:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-18T15:32:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-18T15:32:06.637 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000227125 s, 2.3 MB/s 2024-03-18T15:32:06.638 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.696 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-03-18T15:32:06.751 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-18 15:31:13.693253798 +0000 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-18 15:26:19.460678751 +0000 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-18 15:26:19.460678751 +0000 2024-03-18T15:32:06.752 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-18 15:26:19.460678751 +0000 2024-03-18T15:32:06.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T15:32:06.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:06.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:06.816 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:06.817 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:06.817 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.819 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-18T15:32:06.819 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-18T15:32:06.819 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000214319 s, 2.4 MB/s 2024-03-18T15:32:06.820 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T15:32:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-18T15:32:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-18T15:32:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:06.877 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-03-18T15:32:06.933 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T15:32:06.933 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-18 15:31:13.723253116 +0000 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-18 15:26:19.800670710 +0000 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-18 15:26:19.800670710 +0000 2024-03-18T15:32:06.934 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-18 15:26:19.800670710 +0000 2024-03-18T15:32:06.934 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T15:32:07.000 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-18T15:32:07.000 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-18T15:32:07.000 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000177125 s, 2.9 MB/s 2024-03-18T15:32:07.001 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T15:32:07.057 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-03-18T15:32:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T15:32:07.112 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-18 15:31:13.752252456 +0000 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-18 15:26:20.124663047 +0000 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-18 15:26:20.124663047 +0000 2024-03-18T15:32:07.113 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-18 15:26:20.124663047 +0000 2024-03-18T15:32:07.113 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T15:32:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-03-18T15:32:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-03-18T15:32:07.178 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000173786 s, 2.9 MB/s 2024-03-18T15:32:07.179 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T15:32:07.235 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-18T15:32:07.235 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T15:32:07.254 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T15:32:07.254 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-03-18T15:32:07.311 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-18 15:31:53.851366311 +0000 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-18 15:26:38.043808712 +0000 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-18 15:26:38.043808712 +0000 2024-03-18T15:32:07.312 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-18 15:26:38.043808712 +0000 2024-03-18T15:32:07.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T15:32:07.382 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-18T15:32:07.382 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-18T15:32:07.382 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000267674 s, 1.9 MB/s 2024-03-18T15:32:07.383 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T15:32:07.442 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-18 15:31:53.852366293 +0000 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-18 15:26:38.390801999 +0000 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-18 15:26:38.390801999 +0000 2024-03-18T15:32:07.499 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-18 15:26:38.390801999 +0000 2024-03-18T15:32:07.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T15:32:07.568 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-18T15:32:07.568 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-18T15:32:07.568 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000232011 s, 2.2 MB/s 2024-03-18T15:32:07.569 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T15:32:07.626 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-03-18T15:32:07.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:07 smithi193 ceph-mon[36702]: Reconfiguring mon.c (monmap changed)... 2024-03-18T15:32:07.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:07 smithi193 ceph-mon[36702]: Reconfiguring daemon mon.c on smithi193 2024-03-18T15:32:07.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:07 smithi193 ceph-mon[36702]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi137=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:07.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:07 smithi193 ceph-mon[36702]: Saving service mgr spec with placement smithi096=x;smithi137=y;smithi193=z;count:3 2024-03-18T15:32:07.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:07 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:07.684 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-18 15:31:53.852366293 +0000 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-18 15:26:38.722795576 +0000 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-18 15:26:38.722795576 +0000 2024-03-18T15:32:07.685 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-18 15:26:38.722795576 +0000 2024-03-18T15:32:07.685 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T15:32:07.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:07 smithi137 ceph-mon[35868]: Reconfiguring mon.c (monmap changed)... 2024-03-18T15:32:07.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:07 smithi137 ceph-mon[35868]: Reconfiguring daemon mon.c on smithi193 2024-03-18T15:32:07.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:07 smithi137 ceph-mon[35868]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi137=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:07.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:07 smithi137 ceph-mon[35868]: Saving service mgr spec with placement smithi096=x;smithi137=y;smithi193=z;count:3 2024-03-18T15:32:07.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:07 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:07.753 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-18T15:32:07.753 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-18T15:32:07.753 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000173673 s, 2.9 MB/s 2024-03-18T15:32:07.754 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T15:32:07.811 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:07.867 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-18 15:31:53.852366293 +0000 2024-03-18T15:32:07.868 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-18 15:26:39.058789075 +0000 2024-03-18T15:32:07.868 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-18 15:26:39.058789075 +0000 2024-03-18T15:32:07.868 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-18 15:26:39.058789075 +0000 2024-03-18T15:32:07.868 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T15:32:07.934 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-18T15:32:07.934 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-18T15:32:07.934 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000213381 s, 2.4 MB/s 2024-03-18T15:32:07.936 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T15:32:07.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:07 smithi096 ceph-mon[31930]: Reconfiguring mon.c (monmap changed)... 2024-03-18T15:32:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:07 smithi096 ceph-mon[31930]: Reconfiguring daemon mon.c on smithi193 2024-03-18T15:32:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:07 smithi096 ceph-mon[31930]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi137=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:07 smithi096 ceph-mon[31930]: Saving service mgr spec with placement smithi096=x;smithi137=y;smithi193=z;count:3 2024-03-18T15:32:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:07 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:07.992 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-03-18T15:32:07.992 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T15:32:08.009 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T15:32:08.009 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-18 15:31:44.321576261 +0000 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-18 15:26:37.221045200 +0000 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-18 15:26:37.221045200 +0000 2024-03-18T15:32:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-03-18 15:26:37.221045200 +0000 2024-03-18T15:32:08.065 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T15:32:08.132 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-18T15:32:08.132 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-18T15:32:08.132 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000259457 s, 2.0 MB/s 2024-03-18T15:32:08.133 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T15:32:08.188 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:08.244 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-18 15:31:44.321576261 +0000 2024-03-18T15:32:08.245 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-18 15:26:37.568039167 +0000 2024-03-18T15:32:08.245 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-18 15:26:37.568039167 +0000 2024-03-18T15:32:08.245 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-03-18 15:26:37.568039167 +0000 2024-03-18T15:32:08.245 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T15:32:08.310 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-18T15:32:08.310 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-18T15:32:08.310 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000252023 s, 2.0 MB/s 2024-03-18T15:32:08.311 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T15:32:08.366 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-03-18T15:32:08.422 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-18 15:31:44.321576261 +0000 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-18 15:26:37.899033412 +0000 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-18 15:26:37.899033412 +0000 2024-03-18T15:32:08.423 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-03-18 15:26:37.899033412 +0000 2024-03-18T15:32:08.423 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T15:32:08.488 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-18T15:32:08.488 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-18T15:32:08.489 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000196928 s, 2.6 MB/s 2024-03-18T15:32:08.490 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T15:32:08.546 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-03-18T15:32:08.601 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-03-18 15:31:44.322576245 +0000 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-03-18 15:26:38.258027170 +0000 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-03-18 15:26:38.258027170 +0000 2024-03-18T15:32:08.602 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-03-18 15:26:38.258027170 +0000 2024-03-18T15:32:08.602 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T15:32:08.667 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-03-18T15:32:08.667 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-03-18T15:32:08.668 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000250002 s, 2.0 MB/s 2024-03-18T15:32:08.668 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T15:32:08.725 INFO:tasks.cephadm:Deploying osd.0 on smithi096 with /dev/vg_nvme/lv_4... 2024-03-18T15:32:08.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:08.740 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:08.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:08.961 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:32:09.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:09 smithi096 ceph-mon[31930]: Deploying daemon mgr.z on smithi193 2024-03-18T15:32:09.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:09 smithi137 ceph-mon[35868]: Deploying daemon mgr.z on smithi193 2024-03-18T15:32:09.889 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:09 smithi193 ceph-mon[36702]: Deploying daemon mgr.z on smithi193 2024-03-18T15:32:10.168 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 systemd[1]: Starting Ceph mgr.z for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:32:10.616 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:10 smithi193 ceph-mon[36702]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:10.616 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 podman[38550]: 2024-03-18 15:32:10.331017616 +0000 UTC m=+0.105885233 container create 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-18T15:32:10.617 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 podman[38550]: 2024-03-18 15:32:10.242563718 +0000 UTC m=+0.017431342 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:32:10.617 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 podman[38550]: 2024-03-18 15:32:10.549916414 +0000 UTC m=+0.324784042 container init 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-18T15:32:10.617 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 podman[38550]: 2024-03-18 15:32:10.552487663 +0000 UTC m=+0.327355287 container start 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T15:32:10.617 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 ceph-mgr[38569]: -- 172.21.15.193:0/2888859141 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9e49805a0 con 0x55a9e4955000 2024-03-18T15:32:10.617 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 bash[38550]: 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 2024-03-18T15:32:10.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:10 smithi096 ceph-mon[31930]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:10.722 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T15:32:10.722 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T15:32:10.722 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-18T15:32:10.723 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-18T15:32:10.723 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130635 s, 803 MB/s 2024-03-18T15:32:10.723 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-18T15:32:10.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:10 smithi137 ceph-mon[35868]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:10.917 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:10 smithi193 systemd[1]: Started Ceph mgr.z for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:32:11.198 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:11.104+0000 7f73092da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:32:11.198 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:11.197+0000 7f73092da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:32:11.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-03-18T15:32:11.538 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:11.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:11.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: Deploying daemon mgr.y on smithi137 2024-03-18T15:32:11.932 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:11 smithi096 ceph-mon[31930]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:11.843+0000 7f73092da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:11.935+0000 7f73092da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:11.936 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:11.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:11.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: Deploying daemon mgr.y on smithi137 2024-03-18T15:32:11.937 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:11 smithi193 ceph-mon[36702]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:12.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: Deploying daemon mgr.y on smithi137 2024-03-18T15:32:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:11 smithi137 ceph-mon[35868]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:12.253 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.076+0000 7f73092da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:32:12.254 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.171+0000 7f73092da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:32:12.254 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.253+0000 7f73092da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:32:12.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.515+0000 7f73092da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T15:32:12.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.623+0000 7f73092da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:32:12.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.707+0000 7f73092da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:32:12.803 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.802+0000 7f73092da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:32:12.827 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:12 smithi137 systemd[1]: Starting Ceph mgr.y for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.929+0000 7f09a7c4c700 1 -- 172.21.15.96:0/3271826472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 msgr2=0x7f09a0101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.929+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/3271826472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a0101670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f099c0099c0 tx=0x7f099c009cd0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- 172.21.15.96:0/3271826472 shutdown_connections 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/3271826472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 0x7f09a01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/3271826472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a0101670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/3271826472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 0x7f09a01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- 172.21.15.96:0/3271826472 >> 172.21.15.96:0/3271826472 conn(0x7f09a00fc7f0 msgr2=0x7f09a00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- 172.21.15.96:0/3271826472 shutdown_connections 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- 172.21.15.96:0/3271826472 wait complete. 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 Processor -- start 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- start start 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 0x7f09a0103830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 0x7f09a0103d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010ce90 con 0x7f09a0101250 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010cd10 con 0x7f09a0102450 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.930+0000 7f09a7c4c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a010d010 con 0x7f09a0102df0 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:12.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42496/0 (socket says 172.21.15.96:42496) 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 -- 172.21.15.96:0/2213171353 learned_addr learned my addr 172.21.15.96:0/2213171353 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a61e9700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 0x7f09a0103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a51e7700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 0x7f09a0103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 -- 172.21.15.96:0/2213171353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 msgr2=0x7f09a0103d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 0x7f09a0103d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 -- 172.21.15.96:0/2213171353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 msgr2=0x7f09a0103830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 0x7f09a0103830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 -- 172.21.15.96:0/2213171353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a0104620 con 0x7f09a0101250 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a59e8700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c027070 con 0x7f09a0101250 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c004150 con 0x7f09a0101250 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a01a2610 con 0x7f09a0101250 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c022e00 con 0x7f09a0101250 2024-03-18T15:32:12.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.931+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01a2a60 con 0x7f09a0101250 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.932+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46162+0+0 (secure 0 0 0) 0x7f099c02a070 con 0x7f09a0101250 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.932+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a0068fc0 con 0x7f09a0101250 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.932+0000 7f0996ffd700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 0x7f098c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.932+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f099c04cf20 con 0x7f09a0101250 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.932+0000 7f09a51e7700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 0x7f098c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:12.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.933+0000 7f09a51e7700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 0x7f098c03b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f099000ac50 tx=0x7f099000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:12.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:12.937+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f09a0068fc0 con 0x7f09a0101250 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:12 smithi137 podman[37232]: 2024-03-18 15:32:12.729573821 +0000 UTC m=+0.017835890 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:12 smithi137 podman[37232]: 2024-03-18 15:32:12.828910106 +0000 UTC m=+0.117172217 container create 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 podman[37232]: 2024-03-18 15:32:13.006014341 +0000 UTC m=+0.294276405 container init 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 podman[37232]: 2024-03-18 15:32:13.008633593 +0000 UTC m=+0.296895639 container start 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0) 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 ceph-mgr[37251]: -- 172.21.15.137:0/4073791506 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559afd1f05a0 con 0x559afd1c4c00 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 bash[37232]: 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b 2024-03-18T15:32:13.106 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 systemd[1]: Started Ceph mgr.y for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:32:13.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:13.143+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f09a00fe930 con 0x7f098c039200 2024-03-18T15:32:13.167 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:12.978+0000 7f73092da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:32:13.855 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:13.569+0000 7fd6448d2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:32:13.855 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:13 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:13.665+0000 7fd6448d2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:32:14.307 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.018+0000 7f73092da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.153+0000 7f73092da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:14.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-mon[36702]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:14.335 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:14 smithi096 ceph-mon[31930]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:14.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:14.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-mon[35868]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:14.590 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.499+0000 7f73092da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:32:14.590 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.589+0000 7f73092da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:32:14.693 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:14.353+0000 7fd6448d2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:32:14.693 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:14.447+0000 7fd6448d2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:32:14.693 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:14.596+0000 7fd6448d2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:32:14.693 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:14.691+0000 7fd6448d2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:32:14.918 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.795+0000 7f73092da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:32:15.041 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:14.777+0000 7fd6448d2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:32:15.041 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:15.040+0000 7fd6448d2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-18T15:32:15.266 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:14.981+0000 7f73092da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:32:15.331 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:15.149+0000 7fd6448d2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:32:15.332 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:15.235+0000 7fd6448d2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:32:15.332 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:15.330+0000 7fd6448d2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:32:15.526 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:15.265+0000 7f73092da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:32:15.526 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:15.525+0000 7f73092da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:32:15.605 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:15.509+0000 7fd6448d2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:15.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:15.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:15.764+0000 7f73092da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:15.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:15.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:15.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-18T15:32:15.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:16.855 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:16.477+0000 7fd6448d2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:16.605+0000 7fd6448d2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: Reconfiguring daemon mgr.x on smithi096 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]: dispatch 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]': finished 2024-03-18T15:32:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T15:32:16.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:16.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/257948243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:16.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:16.918 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:16.565+0000 7f73092da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: Reconfiguring daemon mgr.x on smithi096 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]: dispatch 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]': finished 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/257948243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:16.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: Reconfiguring mgr.x (unknown last config time)... 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: Reconfiguring daemon mgr.x on smithi096 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]: dispatch 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1632518289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09dd9fb0-b2af-41c6-bf5b-8c7c01d82677"}]': finished 2024-03-18T15:32:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: osdmap e5: 1 total, 0 up, 1 in 2024-03-18T15:32:16.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:16.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/257948243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:16.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:17.241 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:16 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:16.956+0000 7fd6448d2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:32:17.242 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:17.046+0000 7fd6448d2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:32:17.242 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:17.240+0000 7fd6448d2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:32:17.418 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:17.131+0000 7f73092da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:32:17.418 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:17.230+0000 7f73092da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:32:17.550 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:17.424+0000 7fd6448d2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:32:17.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:17.556+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) ==== 91698+0+0 (secure 0 0 0) 0x7f099c00fd00 con 0x7f09a0101250 2024-03-18T15:32:17.855 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:17.708+0000 7fd6448d2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: Standby manager daemon z started 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T15:32:17.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z[38565]: 2024-03-18T15:32:17.449+0000 7f73092da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: Standby manager daemon z started 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T15:32:17.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:17 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: Standby manager daemon z started 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-18T15:32:17.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:17 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.193:0/977789475' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:18.355 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:17.987+0000 7fd6448d2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:32:18.355 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:18 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:18.222+0000 7fd6448d2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:32:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:18 smithi137 ceph-mon[35868]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-18T15:32:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:18 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T15:32:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:18 smithi193 ceph-mon[36702]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-18T15:32:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:18 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T15:32:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:18 smithi096 ceph-mon[31930]: mgrmap e14: x(active, since 76s), standbys: z 2024-03-18T15:32:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:18 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-18T15:32:19.355 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:19.018+0000 7fd6448d2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:32:19.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:19 smithi096 ceph-mon[31930]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:19.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:19 smithi137 ceph-mon[35868]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:19.856 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:19.587+0000 7fd6448d2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:32:19.856 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:19.685+0000 7fd6448d2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:32:19.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:19 smithi193 ceph-mon[36702]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:20.355 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:32:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y[37247]: 2024-03-18T15:32:19.902+0000 7fd6448d2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:32:20.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:20.571+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f099c003c70 con 0x7f09a0101250 2024-03-18T15:32:20.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:20 smithi096 ceph-mon[31930]: Standby manager daemon y started 2024-03-18T15:32:20.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:20 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T15:32:20.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:20 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:20.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:20 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T15:32:20.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:20 smithi096 ceph-mon[31930]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:20 smithi137 ceph-mon[35868]: Standby manager daemon y started 2024-03-18T15:32:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:20 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T15:32:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:20 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:20.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:20 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T15:32:20.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:20 smithi137 ceph-mon[35868]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:20 smithi193 ceph-mon[36702]: Standby manager daemon y started 2024-03-18T15:32:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:20 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-18T15:32:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:20 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-18T15:32:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:20 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-18T15:32:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:20 smithi193 ceph-mon[36702]: from='mgr.? 172.21.15.137:0/2437095643' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-18T15:32:21.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-18T15:32:21.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T15:32:21.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:21 smithi096 ceph-mon[31930]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:21.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:21 smithi137 ceph-mon[35868]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:21.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:21.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:21.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:21 smithi193 ceph-mon[36702]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:24 smithi193 ceph-mon[36702]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T15:32:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:24 smithi193 ceph-mon[36702]: Deploying daemon osd.0 on smithi096 2024-03-18T15:32:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:24 smithi096 ceph-mon[31930]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T15:32:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:24 smithi096 ceph-mon[31930]: Deploying daemon osd.0 on smithi096 2024-03-18T15:32:24.845 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:24 smithi137 ceph-mon[35868]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:24.845 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-18T15:32:24.845 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:24.845 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:24 smithi137 ceph-mon[35868]: Deploying daemon osd.0 on smithi096 2024-03-18T15:32:26.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:26 smithi096 ceph-mon[31930]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:26.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:26 smithi137 ceph-mon[35868]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:26.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:26 smithi193 ceph-mon[36702]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:28.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:28 smithi096 ceph-mon[31930]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:28.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:28 smithi137 ceph-mon[35868]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:28 smithi193 ceph-mon[36702]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:30.018 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.018 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.018 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:30.018 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:29 smithi096 ceph-mon[31930]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:30.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:30.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:29 smithi193 ceph-mon[36702]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:29 smithi137 ceph-mon[35868]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:31.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:31.859 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:31.859 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:31.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:31.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.212+0000 7f0996ffd700 1 -- 172.21.15.96:0/2213171353 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09a00fe930 con 0x7f098c039200 2024-03-18T15:32:32.213 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 0 on host 'smithi096' 2024-03-18T15:32:32.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 msgr2=0x7f098c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:32.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 0x7f098c03b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f099000ac50 tx=0x7f099000a380 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 msgr2=0x7f09a01051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c009580 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 shutdown_connections 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.214+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f098c039200 0x7f098c03b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a0102df0 0x7f09a0103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a0101250 0x7f09a01051e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 --2- 172.21.15.96:0/2213171353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a0102450 0x7f09a0103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 >> 172.21.15.96:0/2213171353 conn(0x7f09a00fc7f0 msgr2=0x7f09a00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 shutdown_connections 2024-03-18T15:32:32.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:32:32.215+0000 7f09a7c4c700 1 -- 172.21.15.96:0/2213171353 wait complete. 2024-03-18T15:32:32.831 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:32 smithi096 ceph-mon[31930]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:32.832 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.832 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.832 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:32 smithi137 ceph-mon[35868]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:32.857 DEBUG:teuthology.orchestra.run.smithi096:osd.0> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.0.service 2024-03-18T15:32:32.859 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-03-18T15:32:32.860 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T15:32:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:32 smithi193 ceph-mon[36702]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:33.103 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:32:33.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:33 smithi137 ceph-mon[35868]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T15:32:33.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:33 smithi193 ceph-mon[36702]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T15:32:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:33 smithi096 ceph-mon[31930]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-18T15:32:34.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-mon[31930]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:34.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-mon[31930]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T15:32:34.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-mon[31930]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T15:32:34.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-mon[31930]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:32:34.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:34.687 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0[47044]: 2024-03-18T15:32:34.543+0000 7f7f39c97700 -1 osd.0 0 waiting for initial osdmap 2024-03-18T15:32:34.687 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:32:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0[47044]: 2024-03-18T15:32:34.548+0000 7f7f35a8a700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:32:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:34 smithi137 ceph-mon[35868]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:34 smithi137 ceph-mon[35868]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T15:32:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:34 smithi137 ceph-mon[35868]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T15:32:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:34 smithi137 ceph-mon[35868]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:32:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:34 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:34 smithi193 ceph-mon[36702]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:34 smithi193 ceph-mon[36702]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-18T15:32:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:34 smithi193 ceph-mon[36702]: osdmap e6: 1 total, 0 up, 1 in 2024-03-18T15:32:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:34 smithi193 ceph-mon[36702]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:32:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:34 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:34.943 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T15:32:34.943 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T15:32:34.943 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-18T15:32:34.943 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-18T15:32:34.944 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112328 s, 933 MB/s 2024-03-18T15:32:34.944 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-18T15:32:35.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-03-18T15:32:35.740 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:32:35.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:35 smithi096 ceph-mon[31930]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:32:35.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:35 smithi096 ceph-mon[31930]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T15:32:35.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:35.813 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:35.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:35 smithi137 ceph-mon[35868]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:32:35.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:35 smithi137 ceph-mon[35868]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T15:32:35.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:35.854 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:35 smithi193 ceph-mon[36702]: from='osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:32:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:35 smithi193 ceph-mon[36702]: osdmap e7: 1 total, 0 up, 1 in 2024-03-18T15:32:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:32:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] boot 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] boot 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:32:36.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:36.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:36.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: osd.0 [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] boot 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: osdmap e8: 1 total, 1 up, 1 in 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:32:36.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:36.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:36.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:36.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:37.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 -- 172.21.15.137:0/2045875539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41022f0 msgr2=0x7f6ea4102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 --2- 172.21.15.137:0/2045875539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41022f0 0x7f6ea4102710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e940099c0 tx=0x7f6e94009cd0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 -- 172.21.15.137:0/2045875539 shutdown_connections 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 --2- 172.21.15.137:0/2045875539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 0x7f6ea410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 --2- 172.21.15.137:0/2045875539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41022f0 0x7f6ea4102710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 --2- 172.21.15.137:0/2045875539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41034f0 0x7f6ea4103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 -- 172.21.15.137:0/2045875539 >> 172.21.15.137:0/2045875539 conn(0x7f6ea40fd8b0 msgr2=0x7f6ea40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 -- 172.21.15.137:0/2045875539 shutdown_connections 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 -- 172.21.15.137:0/2045875539 wait complete. 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.381+0000 7f6eaa879700 1 Processor -- start 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 -- start start 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41022f0 0x7f6ea4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:37.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 0x7f6ea419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea410deb0 con 0x7f6ea4103e90 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea410dd30 con 0x7f6ea41034f0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6eaa879700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea410e030 con 0x7f6ea41022f0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59162/0 (socket says 172.21.15.137:59162) 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 -- 172.21.15.137:0/3329064168 learned_addr learned my addr 172.21.15.137:0/3329064168 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea3fff700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41022f0 0x7f6ea4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea8e16700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 0x7f6ea419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 -- 172.21.15.137:0/3329064168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41022f0 msgr2=0x7f6ea4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41022f0 0x7f6ea4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 -- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 msgr2=0x7f6ea419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 0x7f6ea419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.382+0000 7f6ea37fe700 1 -- 172.21.15.137:0/3329064168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea41a0060 con 0x7f6ea41034f0 2024-03-18T15:32:37.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6ea37fe700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c9b0 tx=0x7f6e9800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:37.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007b00 con 0x7f6ea41034f0 2024-03-18T15:32:37.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e9800ce50 con 0x7f6ea41034f0 2024-03-18T15:32:37.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea41a0350 con 0x7f6ea41034f0 2024-03-18T15:32:37.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98018660 con 0x7f6ea41034f0 2024-03-18T15:32:37.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.383+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41a08e0 con 0x7f6ea41034f0 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.384+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea4068f90 con 0x7f6ea41034f0 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.385+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f6e98018800 con 0x7f6ea41034f0 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.386+0000 7f6ea17fa700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 0x7f6e8c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.386+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f6e980cd0b0 con 0x7f6ea41034f0 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.386+0000 7f6ea3fff700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 0x7f6e8c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:32:37.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.386+0000 7f6ea3fff700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 0x7f6e8c0a4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e94003890 tx=0x7f6e94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:32:37.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.389+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6e98014030 con 0x7f6ea41034f0 2024-03-18T15:32:37.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:37.593+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ea4062710 con 0x7f6e8c0a2880 2024-03-18T15:32:37.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:37 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi096 2024-03-18T15:32:37.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:37 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi096 to 17088M 2024-03-18T15:32:37.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:37 smithi137 ceph-mon[35868]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T15:32:37.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:37 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi096 2024-03-18T15:32:37.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:37 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi096 to 17088M 2024-03-18T15:32:37.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:37 smithi193 ceph-mon[36702]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T15:32:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:37 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi096 2024-03-18T15:32:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:37 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi096 to 17088M 2024-03-18T15:32:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:37 smithi096 ceph-mon[31930]: osdmap e9: 1 total, 1 up, 1 in 2024-03-18T15:32:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:38 smithi137 ceph-mon[35868]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:38 smithi137 ceph-mon[35868]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:38 smithi193 ceph-mon[36702]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:38 smithi193 ceph-mon[36702]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:38 smithi096 ceph-mon[31930]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:38 smithi096 ceph-mon[31930]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:32:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:32:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:32:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:39.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:39 smithi137 ceph-mon[35868]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:39.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:39 smithi193 ceph-mon[36702]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:39.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:39 smithi096 ceph-mon[31930]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:40 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/923514944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:40 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:40 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]': finished 2024-03-18T15:32:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:40 smithi137 ceph-mon[35868]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T15:32:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:40 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:40 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/923514944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:40 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:40 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]': finished 2024-03-18T15:32:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:40 smithi193 ceph-mon[36702]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T15:32:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:40 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:40 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/923514944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:40 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]: dispatch 2024-03-18T15:32:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:40 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "72c98ebb-ace5-408f-b919-a69839b4c3b6"}]': finished 2024-03-18T15:32:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:40 smithi096 ceph-mon[31930]: osdmap e10: 2 total, 1 up, 2 in 2024-03-18T15:32:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:40 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:41.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:41 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/2946124765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:41.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:41 smithi137 ceph-mon[35868]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:41.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:41 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/2946124765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:41.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:41 smithi193 ceph-mon[36702]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:41.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:41 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/2946124765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:32:41.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:41 smithi096 ceph-mon[31930]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:44 smithi096 ceph-mon[31930]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:44 smithi137 ceph-mon[35868]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:44 smithi193 ceph-mon[36702]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:46 smithi137 ceph-mon[35868]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:46 smithi096 ceph-mon[31930]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:46 smithi193 ceph-mon[36702]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T15:32:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T15:32:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-18T15:32:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:48.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:48 smithi137 ceph-mon[35868]: Deploying daemon osd.1 on smithi137 2024-03-18T15:32:48.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:48 smithi137 ceph-mon[35868]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:48 smithi193 ceph-mon[36702]: Deploying daemon osd.1 on smithi137 2024-03-18T15:32:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:48 smithi193 ceph-mon[36702]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:48 smithi096 ceph-mon[31930]: Deploying daemon osd.1 on smithi137 2024-03-18T15:32:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:48 smithi096 ceph-mon[31930]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:50.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:50 smithi137 ceph-mon[35868]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:50 smithi193 ceph-mon[36702]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:50 smithi096 ceph-mon[31930]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:52.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:52 smithi096 ceph-mon[31930]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:52 smithi137 ceph-mon[35868]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:52 smithi193 ceph-mon[36702]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:54.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:54 smithi137 ceph-mon[35868]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:54 smithi193 ceph-mon[36702]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:54 smithi096 ceph-mon[31930]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:56.158 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-03-18T15:32:56.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.158+0000 7f6ea17fa700 1 -- 172.21.15.137:0/3329064168 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ea4062710 con 0x7f6e8c0a2880 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 msgr2=0x7f6e8c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 0x7f6e8c0a4d40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6e94003890 tx=0x7f6e94005fb0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 msgr2=0x7f6ea4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e9800c9b0 tx=0x7f6e9800ccc0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 shutdown_connections 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.160+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6e8c0a2880 0x7f6e8c0a4d40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ea4103e90 0x7f6ea419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ea41022f0 0x7f6ea4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 --2- 172.21.15.137:0/3329064168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ea41034f0 0x7f6ea4198940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:32:56.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 >> 172.21.15.137:0/3329064168 conn(0x7f6ea40fd8b0 msgr2=0x7f6ea4106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:32:56.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 shutdown_connections 2024-03-18T15:32:56.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:32:56.161+0000 7f6eaa879700 1 -- 172.21.15.137:0/3329064168 wait complete. 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:56.678 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.1.service 2024-03-18T15:32:56.680 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-03-18T15:32:56.681 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:32:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:32:56.925 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:32:57.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:57 smithi137 ceph-mon[35868]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:57.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:57 smithi137 ceph-mon[35868]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:57.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:57 smithi193 ceph-mon[36702]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:57.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:57 smithi193 ceph-mon[36702]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:57.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:57 smithi096 ceph-mon[31930]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:57.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:57 smithi096 ceph-mon[31930]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-mon[35868]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:58.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T15:32:58.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T15:32:58.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:58.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:58 smithi193 ceph-mon[36702]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: from='osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: osdmap e11: 2 total, 1 up, 2 in 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:32:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:58 smithi096 ceph-mon[31930]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128564 s, 816 MB/s 2024-03-18T15:32:58.712 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-18T15:32:59.016 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1[40445]: 2024-03-18T15:32:58.759+0000 7fd3fdf3b700 -1 osd.1 0 waiting for initial osdmap 2024-03-18T15:32:59.016 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:32:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1[40445]: 2024-03-18T15:32:58.762+0000 7fd3f5d26700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:32:59.324 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-03-18T15:32:59.567 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:32:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:32:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T15:32:59.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: from='osd.1 ' entity='osd.1' 2024-03-18T15:32:59.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:32:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: from='osd.1 ' entity='osd.1' 2024-03-18T15:32:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:32:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:32:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: osdmap e12: 2 total, 1 up, 2 in 2024-03-18T15:32:59.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:32:59.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: from='osd.1 ' entity='osd.1' 2024-03-18T15:32:59.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:32:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:33:00.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] boot 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:00.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] boot 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:00.644 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:33:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:33:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: osd.1 [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] boot 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: osdmap e13: 2 total, 2 up, 2 in 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:01.193 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 -- 172.21.15.193:0/1072085673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 msgr2=0x7f0884103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:01.193 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 --2- 172.21.15.193:0/1072085673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884103990 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878009ca0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.193 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 -- 172.21.15.193:0/1072085673 shutdown_connections 2024-03-18T15:33:01.193 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 --2- 172.21.15.193:0/1072085673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884103ed0 0x7f088410a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.193 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 --2- 172.21.15.193:0/1072085673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884102330 0x7f0884102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 --2- 172.21.15.193:0/1072085673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884103990 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 -- 172.21.15.193:0/1072085673 >> 172.21.15.193:0/1072085673 conn(0x7f08840fd8b0 msgr2=0x7f08840ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 -- 172.21.15.193:0/1072085673 shutdown_connections 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.190+0000 7f088984b700 1 -- 172.21.15.193:0/1072085673 wait complete. 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 Processor -- start 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 -- start start 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 0x7f08841986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884103ed0 0x7f088419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f088410e060 con 0x7f0884102330 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f088410dee0 con 0x7f0884103ed0 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f088984b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f088410e1e0 con 0x7f0884103530 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f08827fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:01.194 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f08827fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:42696/0 (socket says 172.21.15.193:42696) 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.191+0000 7f08827fc700 1 -- 172.21.15.193:0/1218473922 learned_addr learned my addr 172.21.15.193:0/1218473922 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08837fe700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884103ed0 0x7f088419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f0882ffd700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 0x7f08841986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 -- 172.21.15.193:0/1218473922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884103ed0 msgr2=0x7f088419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884103ed0 0x7f088419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 -- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 msgr2=0x7f08841986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 0x7f08841986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 -- 172.21.15.193:0/1218473922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08841a0340 con 0x7f0884103530 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f0882ffd700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 0x7f08841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f08827fc700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0878005910 tx=0x7f0878003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878027070 con 0x7f0884103530 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08841a05d0 con 0x7f0884103530 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0878022470 con 0x7f0884103530 2024-03-18T15:33:01.195 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087800f6c0 con 0x7f0884103530 2024-03-18T15:33:01.196 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.192+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08841a0b00 con 0x7f0884103530 2024-03-18T15:33:01.196 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.193+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0884068f90 con 0x7f0884103530 2024-03-18T15:33:01.197 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.194+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f087802a070 con 0x7f0884103530 2024-03-18T15:33:01.197 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.195+0000 7f0888849700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 0x7f086c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:01.197 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.195+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f08780ce320 con 0x7f0884103530 2024-03-18T15:33:01.197 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.195+0000 7f0882ffd700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 0x7f086c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:01.197 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.195+0000 7f0882ffd700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 0x7f086c0a4cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0884103390 tx=0x7f0874008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:01.200 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.199+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f087805e740 con 0x7f0884103530 2024-03-18T15:33:01.404 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:01.404+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0884062710 con 0x7f086c0a2800 2024-03-18T15:33:01.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:01 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi137 2024-03-18T15:33:01.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:01 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-18T15:33:01.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:01 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi137 2024-03-18T15:33:01.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:01 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-18T15:33:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:01 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi137 2024-03-18T15:33:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:01 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi137 to 17080M 2024-03-18T15:33:02.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T15:33:02.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:02.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:02.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:02.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:02.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T15:33:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:02.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: osdmap e14: 2 total, 2 up, 2 in 2024-03-18T15:33:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/1052266829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]': finished 2024-03-18T15:33:04.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T15:33:04.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:04 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:04.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/1052266829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]': finished 2024-03-18T15:33:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T15:33:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:04 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/1052266829' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]: dispatch 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0139a743-3058-40f5-813f-87cbe0a4d29f"}]': finished 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: osdmap e15: 3 total, 2 up, 3 in 2024-03-18T15:33:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:05.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/406628135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/406628135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/406628135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:06 smithi137 ceph-mon[35868]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:06 smithi193 ceph-mon[36702]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:06 smithi096 ceph-mon[31930]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:08.761 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:08 smithi193 ceph-mon[36702]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:08 smithi137 ceph-mon[35868]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:08 smithi096 ceph-mon[31930]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:10.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:10 smithi193 ceph-mon[36702]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:10.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:10 smithi137 ceph-mon[35868]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:10 smithi096 ceph-mon[31930]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:11.754 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T15:33:11.754 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:11.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T15:33:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:11.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-18T15:33:11.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:12.782 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:12 smithi193 ceph-mon[36702]: Deploying daemon osd.2 on smithi193 2024-03-18T15:33:12.782 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:12 smithi193 ceph-mon[36702]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:12 smithi137 ceph-mon[35868]: Deploying daemon osd.2 on smithi193 2024-03-18T15:33:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:12 smithi137 ceph-mon[35868]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:12 smithi096 ceph-mon[31930]: Deploying daemon osd.2 on smithi193 2024-03-18T15:33:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:12 smithi096 ceph-mon[31930]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:14.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:14 smithi193 ceph-mon[36702]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:14 smithi137 ceph-mon[35868]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:14 smithi096 ceph-mon[31930]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:16.788 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:16 smithi193 ceph-mon[36702]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:16.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:16 smithi137 ceph-mon[35868]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:16 smithi096 ceph-mon[31930]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:18.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:18 smithi193 ceph-mon[36702]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:18 smithi137 ceph-mon[35868]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:18 smithi096 ceph-mon[31930]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:19.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.418+0000 7f0888849700 1 -- 172.21.15.193:0/1218473922 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0884062710 con 0x7f086c0a2800 2024-03-18T15:33:19.418 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.420+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 msgr2=0x7f086c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.420+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 0x7f086c0a4cc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0884103390 tx=0x7f0874008040 comp rx=0 tx=0).stop 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.420+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 msgr2=0x7f0884198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.420+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0878005910 tx=0x7f0878003730 comp rx=0 tx=0).stop 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 shutdown_connections 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f086c0a2800 0x7f086c0a4cc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0884103ed0 0x7f088419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0884102330 0x7f08841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.421 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 --2- 172.21.15.193:0/1218473922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0884103530 0x7f0884198c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 >> 172.21.15.193:0/1218473922 conn(0x7f08840fd8b0 msgr2=0x7f08841068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 shutdown_connections 2024-03-18T15:33:19.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:33:19.421+0000 7f088984b700 1 -- 172.21.15.193:0/1218473922 wait complete. 2024-03-18T15:33:20.064 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.2.service 2024-03-18T15:33:20.067 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_3... 2024-03-18T15:33:20.067 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T15:33:20.311 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:33:20.625 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.625 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.625 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:20.625 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.626 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.626 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:20.626 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:20 smithi096 ceph-mon[31930]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:20.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:20 smithi193 ceph-mon[36702]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:20.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:20 smithi137 ceph-mon[35868]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-18T15:33:21.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-mon[36702]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T15:33:21.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-mon[36702]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T15:33:21.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:21.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-mon[36702]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:21.484 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-mon[36702]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:21.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:21 smithi096 ceph-mon[31930]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T15:33:21.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:21 smithi096 ceph-mon[31930]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T15:33:21.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:21.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:21 smithi096 ceph-mon[31930]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:21.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:21 smithi096 ceph-mon[31930]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:21.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:21 smithi137 ceph-mon[35868]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-18T15:33:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:21 smithi137 ceph-mon[35868]: osdmap e16: 3 total, 2 up, 3 in 2024-03-18T15:33:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:21 smithi137 ceph-mon[35868]: from='osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:21.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:21 smithi137 ceph-mon[35868]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:33:22.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2[41865]: 2024-03-18T15:33:21.844+0000 7efcb0eb5700 -1 osd.2 0 waiting for initial osdmap 2024-03-18T15:33:22.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2[41865]: 2024-03-18T15:33:21.848+0000 7efca8ca0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:22.229 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T15:33:22.229 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T15:33:22.230 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-18T15:33:22.230 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-18T15:33:22.230 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130387 s, 804 MB/s 2024-03-18T15:33:22.230 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.524 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:22 smithi096 ceph-mon[31930]: from='osd.2 ' entity='osd.2' 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:22 smithi193 ceph-mon[36702]: from='osd.2 ' entity='osd.2' 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: osdmap e17: 3 total, 2 up, 3 in 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:22.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:22 smithi137 ceph-mon[35868]: from='osd.2 ' entity='osd.2' 2024-03-18T15:33:22.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-03-18T15:33:23.162 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] boot 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.540 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] boot 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:23.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: osd.2 [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] boot 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: osdmap e18: 3 total, 3 up, 3 in 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:23.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:24.531 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi193 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi193 to 17080M 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T15:33:24.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- 172.21.15.96:0/1145636570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 msgr2=0x7f30ac107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 --2- 172.21.15.96:0/1145636570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac107ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3094009990 tx=0x7f3094009ca0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- 172.21.15.96:0/1145636570 shutdown_connections 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 --2- 172.21.15.96:0/1145636570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac108020 0x7f30ac10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 --2- 172.21.15.96:0/1145636570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac102dc0 0x7f30ac1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 --2- 172.21.15.96:0/1145636570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac107ae0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- 172.21.15.96:0/1145636570 >> 172.21.15.96:0/1145636570 conn(0x7f30ac0fc7f0 msgr2=0x7f30ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- 172.21.15.96:0/1145636570 shutdown_connections 2024-03-18T15:33:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- 172.21.15.96:0/1145636570 wait complete. 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 Processor -- start 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.747+0000 7f30b1514700 1 -- start start 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac102dc0 0x7f30ac102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac108020 0x7f30ac1012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac10e000 con 0x7f30ac1056f0 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac10de80 con 0x7f30ac102dc0 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac10e180 con 0x7f30ac108020 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51494/0 (socket says 172.21.15.96:51494) 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 -- 172.21.15.96:0/2323689190 learned_addr learned my addr 172.21.15.96:0/2323689190 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30aaffd700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac102dc0 0x7f30ac102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30ab7fe700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac108020 0x7f30ac1012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 -- 172.21.15.96:0/2323689190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac108020 msgr2=0x7f30ac1012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac108020 0x7f30ac1012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 -- 172.21.15.96:0/2323689190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac102dc0 msgr2=0x7f30ac102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac102dc0 0x7f30ac102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 -- 172.21.15.96:0/2323689190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac101b60 con 0x7f30ac1056f0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a27fc700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3094005910 tx=0x7f30940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094027070 con 0x7f30ac1056f0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac1a27a0 con 0x7f30ac1056f0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.748+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094004210 con 0x7f30ac1056f0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.749+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309400f460 con 0x7f30ac1056f0 2024-03-18T15:33:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.749+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a2c70 con 0x7f30ac1056f0 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.750+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ac068f90 con 0x7f30ac1056f0 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.751+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 137234+0+0 (secure 0 0 0) 0x7f309402a070 con 0x7f30ac1056f0 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.751+0000 7f30a8ff9700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 0x7f30980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.751+0000 7f30aaffd700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 0x7f30980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.751+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f30940ce8a0 con 0x7f30ac1056f0 2024-03-18T15:33:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.752+0000 7f30aaffd700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 0x7f30980a4cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f309c009a10 tx=0x7f309c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:24.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.755+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3094013070 con 0x7f30ac1056f0 2024-03-18T15:33:24.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi193 2024-03-18T15:33:24.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi193 to 17080M 2024-03-18T15:33:24.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:24.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:24.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T15:33:24.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi193 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi193 to 17080M 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: osdmap e19: 3 total, 3 up, 3 in 2024-03-18T15:33:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:33:24.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:24.977+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f30ac0627d0 con 0x7f30980a2800 2024-03-18T15:33:25.436 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51665]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:25.436 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51665]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:25.436 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51665]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:25.436 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51665]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:25.726 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51709]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:25.726 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51709]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:25.726 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51709]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:25.726 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:33:25 smithi096 sudo[51709]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:26.008 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:25 smithi137 sudo[44575]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:26.008 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:25 smithi137 sudo[44575]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:26.009 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:25 smithi137 sudo[44575]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:26.009 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:25 smithi137 sudo[44575]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:26.009 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:26.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:26.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: osdmap e20: 3 total, 3 up, 3 in 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: pgmap v82: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:26.355 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:26 smithi137 sudo[44581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:26.356 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:26 smithi137 sudo[44581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:26.356 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:26 smithi137 sudo[44581]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:26.356 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:33:26 smithi137 sudo[44581]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:26.532 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45955]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:26.532 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45955]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:26.533 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45955]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:26.533 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45955]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:26.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:26 smithi096 sudo[51770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:26.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:26 smithi096 sudo[51770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:26.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:26 smithi096 sudo[51770]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:26.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:26 smithi096 sudo[51770]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:26.918 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45958]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:26.918 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45958]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:26.918 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45958]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:26.918 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:33:26 smithi193 sudo[45958]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:27.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:27.085+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3094003d30 con 0x7f30ac1056f0 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 sudo[51781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 sudo[51781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 sudo[51781]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 sudo[51781]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 ceph-mon[31930]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 ceph-mon[36702]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45961]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45961]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45961]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:27.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45961]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 ceph-mon[35868]: osdmap e21: 3 total, 3 up, 3 in 2024-03-18T15:33:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:27.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:27.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:27.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45964]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:27.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:27 smithi193 sudo[45964]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:28.059 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 sudo[44595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 sudo[44595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 sudo[44595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:27 smithi137 sudo[44595]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 sudo[44598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 sudo[44598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-18T15:33:28.060 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 sudo[44598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 sudo[44598]: pam_unix(sudo:session): session closed for user root 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]: dispatch 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]': finished 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T15:33:28.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]: dispatch 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]': finished 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T15:33:28.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: pgmap v84: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]: dispatch 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3135073487' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e02c4cc-033c-4b5f-944c-18fbf2ef29bf"}]': finished 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: osdmap e22: 4 total, 3 up, 4 in 2024-03-18T15:33:28.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:29.335 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/24667290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:29.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/24667290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:29.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/24667290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:30.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:30 smithi137 ceph-mon[35868]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:30.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:30 smithi193 ceph-mon[36702]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:30.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:30 smithi096 ceph-mon[31930]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:32 smithi137 ceph-mon[35868]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-18T15:33:33.166 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:32 smithi193 ceph-mon[36702]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-18T15:33:33.166 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:32 smithi096 ceph-mon[31930]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-03-18T15:33:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:34 smithi137 ceph-mon[35868]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-18T15:33:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:34 smithi193 ceph-mon[36702]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-18T15:33:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:34 smithi096 ceph-mon[31930]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-03-18T15:33:35.826 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T15:33:35.826 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:35.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T15:33:35.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-18T15:33:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:36 smithi096 ceph-mon[31930]: Deploying daemon osd.3 on smithi096 2024-03-18T15:33:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:36 smithi096 ceph-mon[31930]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-03-18T15:33:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:36 smithi137 ceph-mon[35868]: Deploying daemon osd.3 on smithi096 2024-03-18T15:33:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:36 smithi137 ceph-mon[35868]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-03-18T15:33:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:36 smithi193 ceph-mon[36702]: Deploying daemon osd.3 on smithi096 2024-03-18T15:33:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:36 smithi193 ceph-mon[36702]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 192 KiB/s wr, 12 op/s 2024-03-18T15:33:38.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:38 smithi096 ceph-mon[31930]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T15:33:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:38 smithi137 ceph-mon[35868]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T15:33:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:38 smithi193 ceph-mon[36702]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-18T15:33:40.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:40 smithi096 ceph-mon[31930]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.2 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-03-18T15:33:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:40 smithi137 ceph-mon[35868]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.2 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-03-18T15:33:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:40 smithi193 ceph-mon[36702]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.2 KiB/s rd, 161 KiB/s wr, 10 op/s 2024-03-18T15:33:41.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:41 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:41 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:41 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:41.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:41 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:41 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:41 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:41.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:41.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:42 smithi137 ceph-mon[35868]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T15:33:42.913 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:42 smithi096 ceph-mon[31930]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T15:33:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:42 smithi193 ceph-mon[36702]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-03-18T15:33:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.231+0000 7f30a8ff9700 1 -- 172.21.15.96:0/2323689190 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30ac0627d0 con 0x7f30980a2800 2024-03-18T15:33:44.232 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 msgr2=0x7f30980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 0x7f30980a4cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f309c009a10 tx=0x7f309c009190 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 msgr2=0x7f30ac100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3094005910 tx=0x7f30940047b0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 shutdown_connections 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30980a2800 0x7f30980a4cc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ac108020 0x7f30ac1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30ac102dc0 0x7f30ac102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 --2- 172.21.15.96:0/2323689190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30ac1056f0 0x7f30ac100d70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 >> 172.21.15.96:0/2323689190 conn(0x7f30ac0fc7f0 msgr2=0x7f30ac1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:44.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 shutdown_connections 2024-03-18T15:33:44.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:33:44.234+0000 7f30b1514700 1 -- 172.21.15.96:0/2323689190 wait complete. 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:33:44.809 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.3.service 2024-03-18T15:33:44.811 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-03-18T15:33:44.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T15:33:45.057 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:33:45.464 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:45 smithi137 ceph-mon[35868]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T15:33:45.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:45 smithi193 ceph-mon[36702]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T15:33:45.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:45 smithi096 ceph-mon[31930]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-18T15:33:46.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:46 smithi193 ceph-mon[36702]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T15:33:46.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:46 smithi193 ceph-mon[36702]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T15:33:46.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:46 smithi193 ceph-mon[36702]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:33:46.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:46.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:46 smithi193 ceph-mon[36702]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-mon[31930]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T15:33:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-mon[31930]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T15:33:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-mon[31930]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:33:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-mon[31930]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:46 smithi137 ceph-mon[35868]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-18T15:33:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:46 smithi137 ceph-mon[35868]: osdmap e23: 4 total, 3 up, 4 in 2024-03-18T15:33:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:46 smithi137 ceph-mon[35868]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:33:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:46 smithi137 ceph-mon[35868]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114559 s, 915 MB/s 2024-03-18T15:33:46.944 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-18T15:33:47.114 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:33:46.798+0000 7f79087ae700 -1 osd.3 0 waiting for initial osdmap 2024-03-18T15:33:47.114 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:33:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:33:46.803+0000 7f7900599700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:47.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:47 smithi137 ceph-mon[35868]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:33:47.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:47 smithi137 ceph-mon[35868]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T15:33:47.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:47 smithi137 ceph-mon[35868]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' 2024-03-18T15:33:47.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-03-18T15:33:47.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:47 smithi193 ceph-mon[36702]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:33:47.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:47 smithi193 ceph-mon[36702]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T15:33:47.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:47 smithi193 ceph-mon[36702]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' 2024-03-18T15:33:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:47 smithi096 ceph-mon[31930]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:33:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:47 smithi096 ceph-mon[31930]: osdmap e24: 4 total, 3 up, 4 in 2024-03-18T15:33:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:47 smithi096 ceph-mon[31930]: from='osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437]' entity='osd.3' 2024-03-18T15:33:47.885 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] boot 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:48.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:48.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:33:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:33:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] boot 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:48.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:33:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: osd.3 [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] boot 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: osdmap e25: 4 total, 4 up, 4 in 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:48.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:33:48.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 -- 172.21.15.137:0/337785124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314102da0 msgr2=0x7fc314109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/337785124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314102da0 0x7fc314109650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00b0d0 tx=0x7fc30c00b3e0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 -- 172.21.15.137:0/337785124 shutdown_connections 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/337785124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314102da0 0x7fc314109650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/337785124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314101200 0x7fc314101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/337785124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102400 0x7fc314102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 -- 172.21.15.137:0/337785124 >> 172.21.15.137:0/337785124 conn(0x7fc3140fc7a0 msgr2=0x7fc3140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 -- 172.21.15.137:0/337785124 shutdown_connections 2024-03-18T15:33:49.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.528+0000 7fc31c2d5700 1 -- 172.21.15.137:0/337785124 wait complete. 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 Processor -- start 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 -- start start 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 0x7fc314198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 0x7fc31419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc31410cee0 con 0x7fc314102da0 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc31410cd60 con 0x7fc314101200 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31c2d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc31410d060 con 0x7fc314102400 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60090/0 (socket says 172.21.15.137:60090) 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 -- 172.21.15.137:0/2021852864 learned_addr learned my addr 172.21.15.137:0/2021852864 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a872700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 0x7fc31419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc319870700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 0x7fc314198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 -- 172.21.15.137:0/2021852864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 msgr2=0x7fc314198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 0x7fc314198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 -- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 msgr2=0x7fc31419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 0x7fc31419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a071700 1 -- 172.21.15.137:0/2021852864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3141a02e0 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc319870700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 0x7fc314198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.529+0000 7fc31a872700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 0x7fc31419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc31a071700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b500 tx=0x7fc30400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304015020 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304011d10 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3141a05d0 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30400f2e0 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.530+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3141a0b90 con 0x7fc314101200 2024-03-18T15:33:49.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.531+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc314068f90 con 0x7fc314101200 2024-03-18T15:33:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.532+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc304018020 con 0x7fc314101200 2024-03-18T15:33:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.533+0000 7fc30b7fe700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 0x7fc3000a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.533+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fc3040cd4d0 con 0x7fc314101200 2024-03-18T15:33:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.533+0000 7fc319870700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 0x7fc3000a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.534+0000 7fc319870700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 0x7fc3000a4d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc310007900 tx=0x7fc310008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.537+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc30401f020 con 0x7fc314101200 2024-03-18T15:33:49.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:49 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi096 2024-03-18T15:33:49.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:49 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi096 to 8544M 2024-03-18T15:33:49.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:49 smithi137 ceph-mon[35868]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T15:33:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:49 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi096 2024-03-18T15:33:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:49 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi096 to 8544M 2024-03-18T15:33:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:49 smithi193 ceph-mon[36702]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T15:33:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:49 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi096 2024-03-18T15:33:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:49 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi096 to 8544M 2024-03-18T15:33:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:49 smithi096 ceph-mon[31930]: osdmap e26: 4 total, 4 up, 4 in 2024-03-18T15:33:49.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:33:49.740+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc3140fe9d0 con 0x7fc3000a2890 2024-03-18T15:33:50.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:50 smithi137 ceph-mon[35868]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:33:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:50 smithi137 ceph-mon[35868]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T15:33:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:50.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:50 smithi193 ceph-mon[36702]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:33:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:50 smithi193 ceph-mon[36702]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T15:33:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:50 smithi096 ceph-mon[31930]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:33:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:50 smithi096 ceph-mon[31930]: osdmap e27: 4 total, 4 up, 4 in 2024-03-18T15:33:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:33:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:33:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:33:51.688 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:51 smithi137 ceph-mon[35868]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:51.689 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:51 smithi137 ceph-mon[35868]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T15:33:51.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:51 smithi193 ceph-mon[36702]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:51.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:51 smithi193 ceph-mon[36702]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T15:33:51.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:51 smithi096 ceph-mon[31930]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:33:51.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:51 smithi096 ceph-mon[31930]: osdmap e28: 4 total, 4 up, 4 in 2024-03-18T15:33:52.815 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:52 smithi137 ceph-mon[35868]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:52.815 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:52 smithi137 ceph-mon[35868]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T15:33:52.837 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:52 smithi096 ceph-mon[31930]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:52.837 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:52 smithi096 ceph-mon[31930]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T15:33:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:52 smithi193 ceph-mon[36702]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:52 smithi193 ceph-mon[36702]: osdmap e29: 4 total, 4 up, 4 in 2024-03-18T15:33:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/1121221517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]': finished 2024-03-18T15:33:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T15:33:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:33:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:53 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/2143367447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/1121221517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]': finished 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:33:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:53 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/2143367447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/1121221517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]: dispatch 2024-03-18T15:33:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1281482-ee35-4d0d-be39-618d007e714d"}]': finished 2024-03-18T15:33:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: osdmap e30: 5 total, 4 up, 5 in 2024-03-18T15:33:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:33:53.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:53 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/2143367447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:33:54.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:54 smithi137 ceph-mon[35868]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:54 smithi193 ceph-mon[36702]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:54 smithi096 ceph-mon[31930]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:33:56.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:56 smithi137 ceph-mon[35868]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:33:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:56 smithi193 ceph-mon[36702]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:33:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:56 smithi096 ceph-mon[31930]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:33:58.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:33:58 smithi137 ceph-mon[35868]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:33:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:33:58 smithi193 ceph-mon[36702]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:33:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:33:58 smithi096 ceph-mon[31930]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:00 smithi137 ceph-mon[35868]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:00.779 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T15:34:00.780 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:00 smithi193 ceph-mon[36702]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T15:34:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:00 smithi096 ceph-mon[31930]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-18T15:34:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:01.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:01 smithi137 ceph-mon[35868]: Deploying daemon osd.4 on smithi137 2024-03-18T15:34:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:01 smithi193 ceph-mon[36702]: Deploying daemon osd.4 on smithi137 2024-03-18T15:34:01.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:01 smithi096 ceph-mon[31930]: Deploying daemon osd.4 on smithi137 2024-03-18T15:34:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:02 smithi137 ceph-mon[35868]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:02 smithi193 ceph-mon[36702]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:02 smithi096 ceph-mon[31930]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:04.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:04 smithi137 ceph-mon[35868]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:04 smithi193 ceph-mon[36702]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:04 smithi096 ceph-mon[31930]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:06.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:06 smithi137 ceph-mon[35868]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:06.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:06.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:06.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:06 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:06 smithi193 ceph-mon[36702]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:06 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:06.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:06 smithi096 ceph-mon[31930]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:06.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:06.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:06.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:06 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:08.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:08 smithi137 ceph-mon[35868]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:08 smithi193 ceph-mon[36702]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:08 smithi096 ceph-mon[31930]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:09.106 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-03-18T15:34:09.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.104+0000 7fc30b7fe700 1 -- 172.21.15.137:0/2021852864 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3140fe9d0 con 0x7fc3000a2890 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.107+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 msgr2=0x7fc3000a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.107+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 0x7fc3000a4d50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc310007900 tx=0x7fc310008040 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.107+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 msgr2=0x7fc314198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.107+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc30400b500 tx=0x7fc30400b8c0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 shutdown_connections 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3000a2890 0x7fc3000a4d50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc314102da0 0x7fc31419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc314101200 0x7fc314198680 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 --2- 172.21.15.137:0/2021852864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc314102400 0x7fc314198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 >> 172.21.15.137:0/2021852864 conn(0x7fc3140fc7a0 msgr2=0x7fc3140fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 shutdown_connections 2024-03-18T15:34:09.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:34:09.108+0000 7fc31c2d5700 1 -- 172.21.15.137:0/2021852864 wait complete. 2024-03-18T15:34:09.798 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.4.service 2024-03-18T15:34:09.800 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-03-18T15:34:09.800 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-18T15:34:10.045 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:10.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:10 smithi193 ceph-mon[36702]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:10.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:10 smithi137 ceph-mon[35868]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:10.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:10 smithi096 ceph-mon[31930]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-18T15:34:11.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:11 smithi193 ceph-mon[36702]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T15:34:11.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:11 smithi193 ceph-mon[36702]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T15:34:11.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:11.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:11 smithi193 ceph-mon[36702]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:11 smithi193 ceph-mon[36702]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:11 smithi096 ceph-mon[31930]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T15:34:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:11 smithi096 ceph-mon[31930]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T15:34:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:11 smithi096 ceph-mon[31930]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:11 smithi096 ceph-mon[31930]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-mon[35868]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-18T15:34:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-mon[35868]: osdmap e31: 5 total, 4 up, 5 in 2024-03-18T15:34:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-mon[35868]: from='osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-mon[35868]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:34:11.856 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:34:11.415+0000 7fa7e57ef700 -1 osd.4 0 waiting for initial osdmap 2024-03-18T15:34:11.857 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:34:11 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:34:11.422+0000 7fa7e15e2700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138784 s, 756 MB/s 2024-03-18T15:34:12.023 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-18T15:34:12.562 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:12 smithi137 ceph-mon[35868]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:12.562 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:12 smithi137 ceph-mon[35868]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:34:12.562 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:12 smithi137 ceph-mon[35868]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T15:34:12.562 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.562 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.647 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:12 smithi193 ceph-mon[36702]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:12.647 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:12 smithi193 ceph-mon[36702]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:34:12.647 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:12 smithi193 ceph-mon[36702]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T15:34:12.647 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.647 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.675 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-03-18T15:34:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:12 smithi096 ceph-mon[31930]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:12 smithi096 ceph-mon[31930]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:34:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:12 smithi096 ceph-mon[31930]: osdmap e32: 5 total, 4 up, 5 in 2024-03-18T15:34:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:12.916 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] boot 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:34:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:34:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] boot 2024-03-18T15:34:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T15:34:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: osd.4 [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] boot 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: osdmap e33: 5 total, 5 up, 5 in 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 -- 172.21.15.193:0/3645015463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 msgr2=0x7efe20109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 --2- 172.21.15.193:0/3645015463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe20109650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efe1800b0a0 tx=0x7efe1800b3b0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 -- 172.21.15.193:0/3645015463 shutdown_connections 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 --2- 172.21.15.193:0/3645015463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe20109650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 --2- 172.21.15.193:0/3645015463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20101200 0x7efe20101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 --2- 172.21.15.193:0/3645015463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20102400 0x7efe20102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 -- 172.21.15.193:0/3645015463 >> 172.21.15.193:0/3645015463 conn(0x7efe200fc7c0 msgr2=0x7efe200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:14.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:14 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi137 2024-03-18T15:34:14.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:14 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-18T15:34:14.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:14 smithi193 ceph-mon[36702]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:14.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:14 smithi193 ceph-mon[36702]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 -- 172.21.15.193:0/3645015463 shutdown_connections 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.560+0000 7efe2786f700 1 -- 172.21.15.193:0/3645015463 wait complete. 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 Processor -- start 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 -- start start 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 0x7efe201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20102400 0x7efe20198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010cde0 con 0x7efe20101200 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010cc60 con 0x7efe20102400 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2786f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010cf60 con 0x7efe20102da0 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe25e0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe25e0c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:49040/0 (socket says 172.21.15.193:49040) 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe25e0c700 1 -- 172.21.15.193:0/2654951345 learned_addr learned my addr 172.21.15.193:0/2654951345 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe24e0a700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20102400 0x7efe20198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.561+0000 7efe2560b700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 0x7efe201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 -- 172.21.15.193:0/2654951345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20102400 msgr2=0x7efe20198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20102400 0x7efe20198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 -- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 msgr2=0x7efe201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 0x7efe201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 -- 172.21.15.193:0/2654951345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201a0040 con 0x7efe20102da0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe2560b700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 0x7efe201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe25e0c700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe18009800 tx=0x7efe1800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18021070 con 0x7efe20102da0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201a02d0 con 0x7efe20102da0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe18005520 con 0x7efe20102da0 2024-03-18T15:34:14.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201a0780 con 0x7efe20102da0 2024-03-18T15:34:14.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.562+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1801b8d0 con 0x7efe20102da0 2024-03-18T15:34:14.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.563+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20068f90 con 0x7efe20102da0 2024-03-18T15:34:14.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.565+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7efe18024070 con 0x7efe20102da0 2024-03-18T15:34:14.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.566+0000 7efe167fc700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 0x7efe0c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.566+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7efe1800f020 con 0x7efe20102da0 2024-03-18T15:34:14.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.566+0000 7efe2560b700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 0x7efe0c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.566+0000 7efe2560b700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 0x7efe0c0a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe20102260 tx=0x7efe1000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:14.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.569+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7efe1802b020 con 0x7efe20102da0 2024-03-18T15:34:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:14 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi137 2024-03-18T15:34:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:14 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-18T15:34:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:14 smithi096 ceph-mon[31930]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:14 smithi096 ceph-mon[31930]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T15:34:14.775 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:14.775+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efe200fe880 con 0x7efe0c0a2910 2024-03-18T15:34:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:14 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi137 2024-03-18T15:34:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:14 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi137 to 8540M 2024-03-18T15:34:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:14 smithi137 ceph-mon[35868]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-03-18T15:34:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:14 smithi137 ceph-mon[35868]: osdmap e34: 5 total, 5 up, 5 in 2024-03-18T15:34:15.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:15.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:15.623 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:15 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:15.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:15.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:15.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:15 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:15 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:16.640 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:16 smithi193 ceph-mon[36702]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:16.640 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:16 smithi193 ceph-mon[36702]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:16 smithi137 ceph-mon[35868]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:16 smithi137 ceph-mon[35868]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:16 smithi096 ceph-mon[31930]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:16 smithi096 ceph-mon[31930]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/2228741374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]': finished 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:18.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:18 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/3404599798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/2228741374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]': finished 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:18 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/3404599798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/2228741374' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]: dispatch 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6e210dc-8cbf-4703-adec-d23138828e11"}]': finished 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: osdmap e35: 6 total, 5 up, 6 in 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:18 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/3404599798' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:20 smithi137 ceph-mon[35868]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:20 smithi193 ceph-mon[36702]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:20 smithi096 ceph-mon[31930]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:22.733 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:22 smithi193 ceph-mon[36702]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:22.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:22 smithi096 ceph-mon[31930]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:22.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:22 smithi137 ceph-mon[35868]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:24 smithi193 ceph-mon[36702]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:24 smithi137 ceph-mon[35868]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:24 smithi096 ceph-mon[31930]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:25.779 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T15:34:25.779 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:25.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T15:34:25.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:25.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-18T15:34:25.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:26.499 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:26 smithi193 ceph-mon[36702]: Deploying daemon osd.5 on smithi193 2024-03-18T15:34:26.499 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:26 smithi193 ceph-mon[36702]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:26.846 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:26 smithi137 ceph-mon[35868]: Deploying daemon osd.5 on smithi193 2024-03-18T15:34:26.846 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:26 smithi137 ceph-mon[35868]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:26 smithi096 ceph-mon[31930]: Deploying daemon osd.5 on smithi193 2024-03-18T15:34:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:26 smithi096 ceph-mon[31930]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:28.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:28 smithi193 ceph-mon[36702]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:28.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:28 smithi137 ceph-mon[35868]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:28 smithi096 ceph-mon[31930]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:30 smithi193 ceph-mon[36702]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:30 smithi137 ceph-mon[35868]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:30 smithi096 ceph-mon[31930]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:31.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:31.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:31.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:31.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:32.643 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:32 smithi193 ceph-mon[36702]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:32 smithi137 ceph-mon[35868]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:32 smithi096 ceph-mon[31930]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:33.841 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-03-18T15:34:33.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.840+0000 7efe167fc700 1 -- 172.21.15.193:0/2654951345 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efe200fe880 con 0x7efe0c0a2910 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 msgr2=0x7efe0c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 0x7efe0c0a4dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe20102260 tx=0x7efe1000a680 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 msgr2=0x7efe2019f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efe18009800 tx=0x7efe1800bd70 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 shutdown_connections 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efe0c0a2910 0x7efe0c0a4dd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe20102da0 0x7efe2019f9a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe20101200 0x7efe201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 --2- 172.21.15.193:0/2654951345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe20102400 0x7efe20198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 >> 172.21.15.193:0/2654951345 conn(0x7efe200fc7c0 msgr2=0x7efe200fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 shutdown_connections 2024-03-18T15:34:33.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:34:33.842+0000 7efe2786f700 1 -- 172.21.15.193:0/2654951345 wait complete. 2024-03-18T15:34:34.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:34 smithi193 ceph-mon[36702]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:34.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:34 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:34 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:34 smithi193 ceph-mon[36702]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:34.738 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:34 smithi193 ceph-mon[36702]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:34.769 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.5.service 2024-03-18T15:34:34.772 INFO:tasks.cephadm:Deploying osd.6 on smithi096 with /dev/vg_nvme/lv_2... 2024-03-18T15:34:34.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T15:34:34.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:34 smithi096 ceph-mon[31930]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:34.803 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:34 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.803 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:34 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.803 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:34 smithi096 ceph-mon[31930]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:34.803 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:34 smithi096 ceph-mon[31930]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:34 smithi137 ceph-mon[35868]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:34 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:34 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:34 smithi137 ceph-mon[35868]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:34.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:34 smithi137 ceph-mon[35868]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-18T15:34:35.020 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:34:36.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:35 smithi137 ceph-mon[35868]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:36.147 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T15:34:36.147 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:35 smithi096 ceph-mon[31930]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: osdmap e36: 6 total, 5 up, 6 in 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:35 smithi193 ceph-mon[36702]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:36.528 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:34:36.193+0000 7f299af1a700 -1 osd.5 0 waiting for initial osdmap 2024-03-18T15:34:36.528 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:34:36.199+0000 7f2992d05700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:34:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T15:34:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T15:34:36.961 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-18T15:34:36.961 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-18T15:34:36.962 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113825 s, 921 MB/s 2024-03-18T15:34:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:36.962 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:36 smithi096 ceph-mon[31930]: from='osd.5 ' entity='osd.5' 2024-03-18T15:34:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:34:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T15:34:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T15:34:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:36 smithi137 ceph-mon[35868]: from='osd.5 ' entity='osd.5' 2024-03-18T15:34:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:34:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: osdmap e37: 6 total, 5 up, 6 in 2024-03-18T15:34:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: osdmap e38: 6 total, 5 up, 6 in 2024-03-18T15:34:37.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:36 smithi193 ceph-mon[36702]: from='osd.5 ' entity='osd.5' 2024-03-18T15:34:37.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] boot 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:37.922 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:37 smithi096 ceph-mon[31930]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:37.926 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:34:38.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] boot 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:37 smithi137 ceph-mon[35868]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:38.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: osd.5 [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] boot 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: osdmap e39: 6 total, 6 up, 6 in 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-18T15:34:38.135 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:37 smithi193 ceph-mon[36702]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-18T15:34:39.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T15:34:39.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi193 2024-03-18T15:34:39.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi193 to 8540M 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T15:34:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi193 2024-03-18T15:34:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi193 to 8540M 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.439 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- 172.21.15.96:0/386258642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c1056c0 msgr2=0x7f429c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 --2- 172.21.15.96:0/386258642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c1056c0 0x7f429c107ab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f428c009990 tx=0x7f428c009ca0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- 172.21.15.96:0/386258642 shutdown_connections 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 --2- 172.21.15.96:0/386258642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c107ff0 0x7f429c10a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 --2- 172.21.15.96:0/386258642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c102d90 0x7f429c105180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 --2- 172.21.15.96:0/386258642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c1056c0 0x7f429c107ab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- 172.21.15.96:0/386258642 >> 172.21.15.96:0/386258642 conn(0x7f429c0fc7c0 msgr2=0x7f429c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- 172.21.15.96:0/386258642 shutdown_connections 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- 172.21.15.96:0/386258642 wait complete. 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 Processor -- start 2024-03-18T15:34:39.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.565+0000 7f42a418a700 1 -- start start 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c102d90 0x7f429c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c1056c0 0x7f429c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f429c10df30 con 0x7f429c107ff0 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f429c10ddb0 con 0x7f429c1056c0 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f429c10e0b0 con 0x7f429c102d90 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53932/0 (socket says 172.21.15.96:53932) 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 -- 172.21.15.96:0/130033329 learned_addr learned my addr 172.21.15.96:0/130033329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a1f26700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c102d90 0x7f429c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a1725700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c1056c0 0x7f429c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 -- 172.21.15.96:0/130033329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c102d90 msgr2=0x7f429c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c102d90 0x7f429c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 -- 172.21.15.96:0/130033329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c1056c0 msgr2=0x7f429c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c1056c0 0x7f429c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 -- 172.21.15.96:0/130033329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c1a0120 con 0x7f429c107ff0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a2727700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4294007d70 tx=0x7f429400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294016020 con 0x7f429c107ff0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f429400ad00 con 0x7f429c107ff0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c1a0410 con 0x7f429c107ff0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.566+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4294011600 con 0x7f429c107ff0 2024-03-18T15:34:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.567+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c1a0950 con 0x7f429c107ff0 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.567+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429c068f90 con 0x7f429c107ff0 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.569+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4294019020 con 0x7f429c107ff0 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.569+0000 7f4292ffd700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 0x7f42880a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.569+0000 7f42a1f26700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 0x7f42880a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.569+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f42940ce0b0 con 0x7f429c107ff0 2024-03-18T15:34:39.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.570+0000 7f42a1f26700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 0x7f42880a4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4298005950 tx=0x7f429800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:39.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.573+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4294020020 con 0x7f429c107ff0 2024-03-18T15:34:39.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: osdmap e40: 6 total, 6 up, 6 in 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi193 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi193 to 8540M 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:39.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:39.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:39.777+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f429c062710 con 0x7f42880a2890 2024-03-18T15:34:40.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:40 smithi096 ceph-mon[31930]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T15:34:40.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:40 smithi096 ceph-mon[31930]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:40.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:40 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:40.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:40 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:40.379 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:40 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:40.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:40 smithi193 ceph-mon[36702]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T15:34:40.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:40 smithi193 ceph-mon[36702]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:40.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:40 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:40.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:40 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:40.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:40 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:40.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:40 smithi137 ceph-mon[35868]: osdmap e41: 6 total, 6 up, 6 in 2024-03-18T15:34:40.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:40 smithi137 ceph-mon[35868]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:40.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:40 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:34:40.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:40 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:34:40.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:40 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:41.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:41 smithi193 ceph-mon[36702]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:41.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:41 smithi193 ceph-mon[36702]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T15:34:41.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:41 smithi096 ceph-mon[31930]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:41.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:41 smithi096 ceph-mon[31930]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T15:34:41.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:41 smithi137 ceph-mon[35868]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:34:41.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:41 smithi137 ceph-mon[35868]: osdmap e42: 6 total, 6 up, 6 in 2024-03-18T15:34:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:42 smithi193 ceph-mon[36702]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T15:34:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:42 smithi193 ceph-mon[36702]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:34:42.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:42 smithi096 ceph-mon[31930]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T15:34:42.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:42 smithi096 ceph-mon[31930]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:34:42.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:42 smithi137 ceph-mon[35868]: osdmap e43: 6 total, 6 up, 6 in 2024-03-18T15:34:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:42 smithi137 ceph-mon[35868]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:34:43.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:43 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]: dispatch 2024-03-18T15:34:43.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:43 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]': finished 2024-03-18T15:34:43.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:43 smithi193 ceph-mon[36702]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T15:34:43.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:43 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:34:43.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:43 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/712154268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:43 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]: dispatch 2024-03-18T15:34:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:43 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]': finished 2024-03-18T15:34:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:43 smithi096 ceph-mon[31930]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T15:34:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:43 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:34:43.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:43 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/712154268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:43.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:43 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]: dispatch 2024-03-18T15:34:43.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:43 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/713812817' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd549f23-4564-43ce-8edb-4d5cfad31277"}]': finished 2024-03-18T15:34:43.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:43 smithi137 ceph-mon[35868]: osdmap e44: 7 total, 6 up, 7 in 2024-03-18T15:34:43.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:43 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:34:43.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:43 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/712154268' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:34:44.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:44 smithi193 ceph-mon[36702]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:34:44.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:44 smithi096 ceph-mon[31930]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:34:44.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:44 smithi137 ceph-mon[35868]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:34:46.725 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:46 smithi096 ceph-mon[31930]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:34:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:46 smithi137 ceph-mon[35868]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:34:46.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:46 smithi193 ceph-mon[36702]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:34:48.617 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:48 smithi096 ceph-mon[31930]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:34:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:48 smithi137 ceph-mon[35868]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:34:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:48 smithi193 ceph-mon[36702]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:34:50.535 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:50 smithi096 ceph-mon[31930]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:50.535 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T15:34:50.535 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:50 smithi137 ceph-mon[35868]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T15:34:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:50 smithi193 ceph-mon[36702]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-18T15:34:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:51.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:51 smithi096 ceph-mon[31930]: Deploying daemon osd.6 on smithi096 2024-03-18T15:34:51.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:51 smithi137 ceph-mon[35868]: Deploying daemon osd.6 on smithi096 2024-03-18T15:34:51.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:51 smithi193 ceph-mon[36702]: Deploying daemon osd.6 on smithi096 2024-03-18T15:34:52.763 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:52 smithi096 ceph-mon[31930]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:52 smithi137 ceph-mon[35868]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:52 smithi193 ceph-mon[36702]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:54 smithi096 ceph-mon[31930]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:54 smithi137 ceph-mon[35868]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:54 smithi193 ceph-mon[36702]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:56.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:56.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:56 smithi137 ceph-mon[35868]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:56 smithi193 ceph-mon[36702]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:56 smithi096 ceph-mon[31930]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:58.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:58.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:58.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:58.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:34:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.398+0000 7f4292ffd700 1 -- 172.21.15.96:0/130033329 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f429c062710 con 0x7f42880a2890 2024-03-18T15:34:59.399 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 6 on host 'smithi096' 2024-03-18T15:34:59.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 msgr2=0x7f42880a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:59.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 0x7f42880a4d50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4298005950 tx=0x7f429800a300 comp rx=0 tx=0).stop 2024-03-18T15:34:59.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 msgr2=0x7f429c19fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4294007d70 tx=0x7f429400a340 comp rx=0 tx=0).stop 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 shutdown_connections 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42880a2890 0x7f42880a4d50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f429c107ff0 0x7f429c19fa80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f429c102d90 0x7f429c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.400+0000 7f42a418a700 1 --2- 172.21.15.96:0/130033329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f429c1056c0 0x7f429c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.401+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 >> 172.21.15.96:0/130033329 conn(0x7f429c0fc7c0 msgr2=0x7f429c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.401+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 shutdown_connections 2024-03-18T15:34:59.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:34:59.401+0000 7f42a418a700 1 -- 172.21.15.96:0/130033329 wait complete. 2024-03-18T15:34:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:59 smithi096 ceph-mon[31930]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T15:34:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:34:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:59.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:59 smithi137 ceph-mon[35868]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T15:34:59.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:34:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:34:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:59 smithi193 ceph-mon[36702]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-18T15:34:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:34:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:34:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:00.040 DEBUG:teuthology.orchestra.run.smithi096:osd.6> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.6.service 2024-03-18T15:35:00.042 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-03-18T15:35:00.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T15:35:00.289 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:35:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-mon[31930]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-mon[31930]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T15:35:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-mon[31930]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T15:35:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-mon[31930]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:35:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:00.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:00 smithi137 ceph-mon[35868]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:00.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:00 smithi137 ceph-mon[35868]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T15:35:00.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:00 smithi137 ceph-mon[35868]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T15:35:00.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:00 smithi137 ceph-mon[35868]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:35:00.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:00.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:00 smithi193 ceph-mon[36702]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:00.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:00 smithi193 ceph-mon[36702]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-18T15:35:00.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:00 smithi193 ceph-mon[36702]: osdmap e45: 7 total, 6 up, 7 in 2024-03-18T15:35:00.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:00 smithi193 ceph-mon[36702]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:35:00.819 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:35:00.901+0000 7f122c6a6700 -1 osd.6 0 waiting for initial osdmap 2024-03-18T15:35:01.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:35:00 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:35:00.912+0000 7f1223c90700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:35:01.542 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:01 smithi096 ceph-mon[31930]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:35:01.542 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:01 smithi096 ceph-mon[31930]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T15:35:01.542 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.543 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.543 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:01 smithi096 ceph-mon[31930]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' 2024-03-18T15:35:01.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:01 smithi137 ceph-mon[35868]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:35:01.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:01 smithi137 ceph-mon[35868]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T15:35:01.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:01 smithi137 ceph-mon[35868]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' 2024-03-18T15:35:01.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:01 smithi193 ceph-mon[36702]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:35:01.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:01 smithi193 ceph-mon[36702]: osdmap e46: 7 total, 6 up, 7 in 2024-03-18T15:35:01.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:01.833 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:01 smithi193 ceph-mon[36702]: from='osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146]' entity='osd.6' 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112345 s, 933 MB/s 2024-03-18T15:35:02.413 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-18T15:35:02.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:35:02.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:35:02.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:02.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:02.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] boot 2024-03-18T15:35:02.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T15:35:02.679 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:02.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:35:02.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:35:02.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:02.717 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:02.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] boot 2024-03-18T15:35:02.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T15:35:02.718 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: osd.6 [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] boot 2024-03-18T15:35:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: osdmap e47: 7 total, 7 up, 7 in 2024-03-18T15:35:02.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-18T15:35:03.110 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-03-18T15:35:03.359 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:35:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T15:35:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi096 2024-03-18T15:35:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi096 to 5696M 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:03 smithi137 ceph-mon[35868]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi096 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi096 to 5696M 2024-03-18T15:35:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:03 smithi193 ceph-mon[36702]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: osdmap e48: 7 total, 7 up, 7 in 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi096 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi096 to 5696M 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:04.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:03 smithi096 ceph-mon[31930]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.068+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1512835376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 msgr2=0x7f0cac10a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.068+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1512835376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac10a770 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b0d0 tx=0x7f0ca400b3e0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1512835376 shutdown_connections 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1512835376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac10a770 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1512835376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 0x7f0cac102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1512835376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac103520 0x7f0cac103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1512835376 >> 172.21.15.137:0/1512835376 conn(0x7f0cac0fd8e0 msgr2=0x7f0cac0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1512835376 shutdown_connections 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1512835376 wait complete. 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 Processor -- start 2024-03-18T15:35:05.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.069+0000 7f0cb0abf700 1 -- start start 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 0x7f0cac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac103520 0x7f0cac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10df10 con 0x7f0cac103520 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10dd90 con 0x7f0cac103ec0 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac10e090 con 0x7f0cac102320 2024-03-18T15:35:05.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34610/0 (socket says 172.21.15.137:34610) 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 -- 172.21.15.137:0/1954374064 learned_addr learned my addr 172.21.15.137:0/1954374064 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caa59c700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 0x7f0cac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 -- 172.21.15.137:0/1954374064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 msgr2=0x7f0cac198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 0x7f0cac198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 -- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac103520 msgr2=0x7f0cac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac103520 0x7f0cac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 -- 172.21.15.137:0/1954374064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac1a0070 con 0x7f0cac103ec0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0caad9d700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40062a0 tx=0x7f0ca4007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4003c00 con 0x7f0cac103ec0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac1a0300 con 0x7f0cac103ec0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca4003da0 con 0x7f0cac103ec0 2024-03-18T15:35:05.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4005560 con 0x7f0cac103ec0 2024-03-18T15:35:05.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.070+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac1a0830 con 0x7f0cac103ec0 2024-03-18T15:35:05.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.071+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cac068f90 con 0x7f0cac103ec0 2024-03-18T15:35:05.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.072+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0ca4005700 con 0x7f0cac103ec0 2024-03-18T15:35:05.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.073+0000 7f0c9b7fe700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 0x7f0c940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:05.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.073+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f0ca400f020 con 0x7f0cac103ec0 2024-03-18T15:35:05.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.073+0000 7f0caa59c700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 0x7f0c940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:05.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.074+0000 7f0caa59c700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 0x7f0c940a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c005fd0 tx=0x7f0c9c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.077+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ca402b020 con 0x7f0cac103ec0 2024-03-18T15:35:05.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:05.283+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0cac0627d0 con 0x7f0c940a2890 2024-03-18T15:35:05.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:05.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:05.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:05.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:05.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:05.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:06.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:06 smithi137 ceph-mon[35868]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:06.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:06 smithi137 ceph-mon[35868]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:06 smithi193 ceph-mon[36702]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:06 smithi193 ceph-mon[36702]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:06.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:06 smithi096 ceph-mon[31930]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:06.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:06 smithi096 ceph-mon[31930]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/805359016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]': finished 2024-03-18T15:35:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T15:35:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/805359016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]': finished 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T15:35:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/805359016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]: dispatch 2024-03-18T15:35:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01fdc4fe-0b84-4109-a831-2b38ba820711"}]': finished 2024-03-18T15:35:08.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: osdmap e49: 8 total, 7 up, 8 in 2024-03-18T15:35:08.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:09.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:09 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/958282062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:09.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:09 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/958282062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:09.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:09 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/958282062' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:10.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:10 smithi137 ceph-mon[35868]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:10 smithi193 ceph-mon[36702]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:10 smithi096 ceph-mon[31930]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:12 smithi137 ceph-mon[35868]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:12 smithi193 ceph-mon[36702]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:12 smithi096 ceph-mon[31930]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:14.849 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:14 smithi137 ceph-mon[35868]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:14.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:14 smithi193 ceph-mon[36702]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:14 smithi096 ceph-mon[31930]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:16 smithi137 ceph-mon[35868]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T15:35:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:16 smithi137 ceph-mon[35868]: Deploying daemon osd.7 on smithi137 2024-03-18T15:35:16.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:16 smithi193 ceph-mon[36702]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T15:35:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:16 smithi193 ceph-mon[36702]: Deploying daemon osd.7 on smithi137 2024-03-18T15:35:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:16 smithi096 ceph-mon[31930]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-18T15:35:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:16 smithi096 ceph-mon[31930]: Deploying daemon osd.7 on smithi137 2024-03-18T15:35:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:18 smithi137 ceph-mon[35868]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:18 smithi193 ceph-mon[36702]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:18 smithi096 ceph-mon[31930]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:20.702 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:20 smithi137 ceph-mon[35868]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:20 smithi193 ceph-mon[36702]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:20.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:20 smithi096 ceph-mon[31930]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:22.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:22.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:22 smithi137 ceph-mon[35868]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:22.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:22 smithi193 ceph-mon[36702]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:22.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:22.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:22.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:22 smithi096 ceph-mon[31930]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:24.841 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:24 smithi137 ceph-mon[35868]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:24.905 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-03-18T15:35:24.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.904+0000 7f0c9b7fe700 1 -- 172.21.15.137:0/1954374064 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0cac0627d0 con 0x7f0c940a2890 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 msgr2=0x7f0c940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 0x7f0c940a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c005fd0 tx=0x7f0c9c00a560 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 msgr2=0x7f0cac19f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40062a0 tx=0x7f0ca4007e40 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 shutdown_connections 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0c940a2890 0x7f0c940a4d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cac103ec0 0x7f0cac19f9d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0cac102320 0x7f0cac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 --2- 172.21.15.137:0/1954374064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac103520 0x7f0cac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 >> 172.21.15.137:0/1954374064 conn(0x7f0cac0fd8e0 msgr2=0x7f0cac106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:24.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 shutdown_connections 2024-03-18T15:35:24.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:35:24.906+0000 7f0cb0abf700 1 -- 172.21.15.137:0/1954374064 wait complete. 2024-03-18T15:35:24.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:24 smithi193 ceph-mon[36702]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:24 smithi096 ceph-mon[31930]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:25.665 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.7.service 2024-03-18T15:35:25.667 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-03-18T15:35:25.668 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-18T15:35:25.916 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.039 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:26.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:26 smithi137 ceph-mon[35868]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T15:35:27.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:26 smithi137 ceph-mon[35868]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T15:35:27.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:26 smithi137 ceph-mon[35868]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:26 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:27.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:26 smithi137 ceph-mon[35868]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:26 smithi193 ceph-mon[36702]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T15:35:27.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:26 smithi193 ceph-mon[36702]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T15:35:27.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:26 smithi193 ceph-mon[36702]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:26 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:27.140 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:26 smithi193 ceph-mon[36702]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:26 smithi096 ceph-mon[31930]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-18T15:35:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:26 smithi096 ceph-mon[31930]: osdmap e50: 8 total, 7 up, 8 in 2024-03-18T15:35:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:26 smithi096 ceph-mon[31930]: from='osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:26 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:26 smithi096 ceph-mon[31930]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:35:27.606 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:35:27.295+0000 7f3a48c52700 -1 osd.7 0 waiting for initial osdmap 2024-03-18T15:35:27.606 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:35:27.307+0000 7f3a40a3d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: from='osd.7 ' entity='osd.7' 2024-03-18T15:35:27.932 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:27 smithi137 ceph-mon[35868]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111933 s, 937 MB/s 2024-03-18T15:35:28.022 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: from='osd.7 ' entity='osd.7' 2024-03-18T15:35:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:27 smithi193 ceph-mon[36702]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: osdmap e51: 8 total, 7 up, 8 in 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: from='osd.7 ' entity='osd.7' 2024-03-18T15:35:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:27 smithi096 ceph-mon[31930]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-18T15:35:28.708 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-03-18T15:35:28.946 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] boot 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T15:35:29.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] boot 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T15:35:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: osd.7 [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] boot 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: osdmap e52: 8 total, 8 up, 8 in 2024-03-18T15:35:29.236 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-18T15:35:30.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T15:35:30.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:30.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.060 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.061 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: osdmap e53: 8 total, 8 up, 8 in 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:30.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:30.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:30.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3885357905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 msgr2=0x7fdb04103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:30.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3885357905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04103950 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009990 tx=0x7fdaf8009ca0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3885357905 shutdown_connections 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3885357905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb04103e90 0x7fdb0410a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3885357905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041022f0 0x7fdb04102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3885357905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04103950 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3885357905 >> 172.21.15.193:0/3885357905 conn(0x7fdb040fd8b0 msgr2=0x7fdb040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3885357905 shutdown_connections 2024-03-18T15:35:30.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.572+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3885357905 wait complete. 2024-03-18T15:35:30.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 Processor -- start 2024-03-18T15:35:30.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 -- start start 2024-03-18T15:35:30.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 0x7fdb04198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:30.576 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103e90 0x7fdb0419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410dee0 con 0x7fdb041022f0 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410dd60 con 0x7fdb04103e90 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb08dc9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0410e060 con 0x7fdb041034f0 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:49906/0 (socket says 172.21.15.193:49906) 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 -- 172.21.15.193:0/3878530950 learned_addr learned my addr 172.21.15.193:0/3878530950 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb02d9d700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103e90 0x7fdb0419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb0259c700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 0x7fdb04198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 -- 172.21.15.193:0/3878530950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103e90 msgr2=0x7fdb0419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103e90 0x7fdb0419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 -- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 msgr2=0x7fdb04198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 0x7fdb04198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb01d9b700 1 -- 172.21.15.193:0/3878530950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb041a0180 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.573+0000 7fdb0259c700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 0x7fdb04198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdb01d9b700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004790 tx=0x7fdaf8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8027070 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb041a0410 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf8004150 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800f460 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.574+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb041a08c0 con 0x7fdb041034f0 2024-03-18T15:35:30.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.575+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb04068f90 con 0x7fdb041034f0 2024-03-18T15:35:30.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.577+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdaf802a070 con 0x7fdb041034f0 2024-03-18T15:35:30.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.577+0000 7fdaf37fe700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:30.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.577+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fdaf80cf540 con 0x7fdb041034f0 2024-03-18T15:35:30.580 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.577+0000 7fdb0259c700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:30.580 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.578+0000 7fdb0259c700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 0x7fdaec0a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005950 tx=0x7fdaf4005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:30.582 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.581+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdaf8031070 con 0x7fdb041034f0 2024-03-18T15:35:30.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:30.785+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdb04062710 con 0x7fdaec0a2890 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi137 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi137 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:31.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi137 2024-03-18T15:35:31.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi137 to 5693M 2024-03-18T15:35:31.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: osdmap e54: 8 total, 8 up, 8 in 2024-03-18T15:35:31.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:31.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:31.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:32.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:31 smithi193 ceph-mon[36702]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:32.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:31 smithi193 ceph-mon[36702]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T15:35:32.095 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:31 smithi193 ceph-mon[36702]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:31 smithi096 ceph-mon[31930]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:31 smithi096 ceph-mon[31930]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T15:35:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:31 smithi096 ceph-mon[31930]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:32.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:31 smithi137 ceph-mon[35868]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:32.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:31 smithi137 ceph-mon[35868]: osdmap e55: 8 total, 8 up, 8 in 2024-03-18T15:35:32.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:31 smithi137 ceph-mon[35868]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:32 smithi193 ceph-mon[36702]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T15:35:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:32 smithi096 ceph-mon[31930]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T15:35:33.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:32 smithi137 ceph-mon[35868]: osdmap e56: 8 total, 8 up, 8 in 2024-03-18T15:35:34.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/4081019092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]': finished 2024-03-18T15:35:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T15:35:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:34.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:34.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/4081019092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]': finished 2024-03-18T15:35:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T15:35:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:33 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/4081019092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]: dispatch 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db47c340-59a5-4202-8488-b5548f9d18fc"}]': finished 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: osdmap e57: 9 total, 8 up, 9 in 2024-03-18T15:35:34.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:33 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:34 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/3900764135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:35.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:34 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/3900764135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:35.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:34 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/3900764135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:35:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:35 smithi096 ceph-mon[31930]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:36.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:35 smithi137 ceph-mon[35868]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:36.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:35 smithi193 ceph-mon[36702]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:38.719 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:38 smithi193 ceph-mon[36702]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:38 smithi137 ceph-mon[35868]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:38 smithi096 ceph-mon[31930]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:40.817 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:40 smithi193 ceph-mon[36702]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:40 smithi137 ceph-mon[35868]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:40 smithi096 ceph-mon[31930]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:41.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:41 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T15:35:41.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:41 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:41.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:41 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T15:35:41.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:41 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:41.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-18T15:35:41.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:41 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:42.771 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:42 smithi193 ceph-mon[36702]: Deploying daemon osd.8 on smithi193 2024-03-18T15:35:42.772 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:42 smithi193 ceph-mon[36702]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:42 smithi137 ceph-mon[35868]: Deploying daemon osd.8 on smithi193 2024-03-18T15:35:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:42 smithi137 ceph-mon[35868]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:42 smithi096 ceph-mon[31930]: Deploying daemon osd.8 on smithi193 2024-03-18T15:35:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:42 smithi096 ceph-mon[31930]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:44 smithi193 ceph-mon[36702]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:44 smithi137 ceph-mon[35868]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:44 smithi096 ceph-mon[31930]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:46.750 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:46 smithi193 ceph-mon[36702]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:46.750 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.751 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.751 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:46 smithi137 ceph-mon[35868]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:46 smithi096 ceph-mon[31930]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:48 smithi193 ceph-mon[36702]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:48 smithi137 ceph-mon[35868]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:48 smithi096 ceph-mon[31930]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:50.054 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-03-18T15:35:50.054 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.052+0000 7fdaf37fe700 1 -- 172.21.15.193:0/3878530950 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdb04062710 con 0x7fdaec0a2890 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 msgr2=0x7fdaec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 0x7fdaec0a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005950 tx=0x7fdaf4005f50 comp rx=0 tx=0).stop 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 msgr2=0x7fdb04198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004790 tx=0x7fdaf8009580 comp rx=0 tx=0).stop 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 shutdown_connections 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdaec0a2890 0x7fdaec0a4d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:50.056 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103e90 0x7fdb0419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:50.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb041022f0 0x7fdb04198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:50.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 --2- 172.21.15.193:0/3878530950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb041034f0 0x7fdb04198a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:50.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 >> 172.21.15.193:0/3878530950 conn(0x7fdb040fd8b0 msgr2=0x7fdb04106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:50.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 shutdown_connections 2024-03-18T15:35:50.057 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:35:50.055+0000 7fdb08dc9700 1 -- 172.21.15.193:0/3878530950 wait complete. 2024-03-18T15:35:50.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:50 smithi137 ceph-mon[35868]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:50 smithi193 ceph-mon[36702]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:50.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:35:50.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:50 smithi096 ceph-mon[31930]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-18T15:35:50.765 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.8.service 2024-03-18T15:35:50.767 INFO:tasks.cephadm:Deploying osd.9 on smithi096 with /dev/vg_nvme/lv_1... 2024-03-18T15:35:50.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T15:35:51.016 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:35:52.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T15:35:52.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.307 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T15:35:52.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:52.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.308 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:52 smithi096 ceph-mon[31930]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.309 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-mon[36702]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:52.310 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:35:52.056+0000 7f6e072fc700 -1 osd.8 0 waiting for initial osdmap 2024-03-18T15:35:52.310 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:35:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:35:52.066+0000 7f6e030ef700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:35:52.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-18T15:35:52.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: from='osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: osdmap e58: 9 total, 8 up, 9 in 2024-03-18T15:35:52.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:52.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:35:52.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:52 smithi137 ceph-mon[35868]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:53.009 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T15:35:53.009 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T15:35:53.009 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-03-18T15:35:53.010 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-03-18T15:35:53.010 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112011 s, 936 MB/s 2024-03-18T15:35:53.010 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-03-18T15:35:53.311 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:53 smithi096 ceph-mon[31930]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:35:53.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:53 smithi096 ceph-mon[31930]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T15:35:53.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.312 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:53 smithi137 ceph-mon[35868]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:35:53.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:53 smithi137 ceph-mon[35868]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T15:35:53.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:53 smithi193 ceph-mon[36702]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:35:53.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:53 smithi193 ceph-mon[36702]: osdmap e59: 9 total, 8 up, 9 in 2024-03-18T15:35:53.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:53.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-03-18T15:35:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] boot 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:54.357 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:54 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] boot 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.370 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:54.371 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:54 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: osd.8 [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] boot 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: osdmap e60: 9 total, 9 up, 9 in 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:35:54.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:54 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:35:55.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:55 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi193 2024-03-18T15:35:55.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:55 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi193 to 5693M 2024-03-18T15:35:55.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:55 smithi137 ceph-mon[35868]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T15:35:55.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:55 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi193 2024-03-18T15:35:55.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:55 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi193 to 5693M 2024-03-18T15:35:55.381 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:55 smithi096 ceph-mon[31930]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T15:35:55.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:55 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi193 2024-03-18T15:35:55.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:55 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi193 to 5693M 2024-03-18T15:35:55.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:55 smithi193 ceph-mon[36702]: osdmap e61: 9 total, 9 up, 9 in 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3865147150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d41056b0 msgr2=0x7fa6d4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3865147150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d41056b0 0x7fa6d4107aa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009990 tx=0x7fa6c8009ca0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3865147150 shutdown_connections 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3865147150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4107fe0 0x7fa6d410a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3865147150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d4102d80 0x7fa6d4105170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3865147150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d41056b0 0x7fa6d4107aa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3865147150 >> 172.21.15.96:0/3865147150 conn(0x7fa6d40fc790 msgr2=0x7fa6d40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3865147150 shutdown_connections 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.748+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3865147150 wait complete. 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 Processor -- start 2024-03-18T15:35:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 -- start start 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4102d80 0x7fa6d41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d41056b0 0x7fa6d4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410df40 con 0x7fa6d4107fe0 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410ddc0 con 0x7fa6d4102d80 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d8a0f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d410e0c0 con 0x7fa6d41056b0 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34140/0 (socket says 172.21.15.96:34140) 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 -- 172.21.15.96:0/3354631131 learned_addr learned my addr 172.21.15.96:0/3354631131 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d259c700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4102d80 0x7fa6d41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d1d9b700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d41056b0 0x7fa6d4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 -- 172.21.15.96:0/3354631131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d41056b0 msgr2=0x7fa6d4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d41056b0 0x7fa6d4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 -- 172.21.15.96:0/3354631131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4102d80 msgr2=0x7fa6d41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4102d80 0x7fa6d41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.749+0000 7fa6d2d9d700 1 -- 172.21.15.96:0/3354631131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d4100b70 con 0x7fa6d4107fe0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6d2d9d700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c900 tx=0x7fa6c400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4004460 con 0x7fa6d4107fe0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c400ce50 con 0x7fa6d4107fe0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003d60 con 0x7fa6d4107fe0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d4100e60 con 0x7fa6d4107fe0 2024-03-18T15:35:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.750+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d4101420 con 0x7fa6d4107fe0 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.751+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d4068f90 con 0x7fa6d4107fe0 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.752+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa6c40076b0 con 0x7fa6d4107fe0 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.752+0000 7fa6c37fe700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.752+0000 7fa6d259c700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.752+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7fa6c40cdf70 con 0x7fa6d4107fe0 2024-03-18T15:35:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.753+0000 7fa6d259c700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00a400 tx=0x7fa6cc00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.757+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa6c4013030 con 0x7fa6d4107fe0 2024-03-18T15:35:55.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:35:55.963+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa6d40fdeb0 con 0x7fa6bc0a2910 2024-03-18T15:35:56.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:56 smithi137 ceph-mon[35868]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T15:35:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:56 smithi137 ceph-mon[35868]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:56.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:56.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:56 smithi193 ceph-mon[36702]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T15:35:56.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:56 smithi193 ceph-mon[36702]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:56.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:56.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:56.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:56.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:56 smithi096 ceph-mon[31930]: osdmap e62: 9 total, 9 up, 9 in 2024-03-18T15:35:56.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:56 smithi096 ceph-mon[31930]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:56.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:35:56.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:35:56.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:35:57.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:57 smithi096 ceph-mon[31930]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:57.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:57 smithi096 ceph-mon[31930]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T15:35:57.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:57 smithi137 ceph-mon[35868]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:57.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:57 smithi137 ceph-mon[35868]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T15:35:57.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:57 smithi193 ceph-mon[36702]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:35:57.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:57 smithi193 ceph-mon[36702]: osdmap e63: 9 total, 9 up, 9 in 2024-03-18T15:35:58.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:58 smithi137 ceph-mon[35868]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T15:35:58.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:58 smithi137 ceph-mon[35868]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:58.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:58 smithi193 ceph-mon[36702]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T15:35:58.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:58 smithi193 ceph-mon[36702]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:58.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:58 smithi096 ceph-mon[31930]: osdmap e64: 9 total, 9 up, 9 in 2024-03-18T15:35:58.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:58 smithi096 ceph-mon[31930]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:35:59.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:59 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]: dispatch 2024-03-18T15:35:59.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:59 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]': finished 2024-03-18T15:35:59.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:59 smithi137 ceph-mon[35868]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T15:35:59.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:35:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:35:59.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:59 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]: dispatch 2024-03-18T15:35:59.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:59 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]': finished 2024-03-18T15:35:59.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:59 smithi193 ceph-mon[36702]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T15:35:59.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:35:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:35:59.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:59 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]: dispatch 2024-03-18T15:35:59.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:59 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4268617123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "214915c3-c5ad-42b8-8c12-04ecc68c6109"}]': finished 2024-03-18T15:35:59.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:59 smithi096 ceph-mon[31930]: osdmap e65: 10 total, 9 up, 10 in 2024-03-18T15:35:59.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:35:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:00.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:00 smithi193 ceph-mon[36702]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-18T15:36:00.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:00 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/646368103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:00.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:00 smithi096 ceph-mon[31930]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-18T15:36:00.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:00 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/646368103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:00.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:00 smithi137 ceph-mon[35868]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-18T15:36:00.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:00 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/646368103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:02 smithi096 ceph-mon[31930]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:02 smithi137 ceph-mon[35868]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:02 smithi193 ceph-mon[36702]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:04.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:04 smithi096 ceph-mon[31930]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:04.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:04 smithi137 ceph-mon[35868]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:04 smithi193 ceph-mon[36702]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:06.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:06 smithi137 ceph-mon[35868]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T15:36:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:06 smithi193 ceph-mon[36702]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T15:36:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:06 smithi096 ceph-mon[31930]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-18T15:36:07.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:07 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T15:36:07.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:07 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:07.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:07 smithi096 ceph-mon[31930]: Deploying daemon osd.9 on smithi096 2024-03-18T15:36:07.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:07 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T15:36:07.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:07 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:07.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:07 smithi137 ceph-mon[35868]: Deploying daemon osd.9 on smithi096 2024-03-18T15:36:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:07 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-18T15:36:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:07 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:07 smithi193 ceph-mon[36702]: Deploying daemon osd.9 on smithi096 2024-03-18T15:36:08.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:08 smithi096 ceph-mon[31930]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T15:36:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:08 smithi137 ceph-mon[35868]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T15:36:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:08 smithi193 ceph-mon[36702]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-18T15:36:10.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:10 smithi137 ceph-mon[35868]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T15:36:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:10 smithi193 ceph-mon[36702]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T15:36:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:10 smithi096 ceph-mon[31930]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-03-18T15:36:12.784 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:12 smithi096 ceph-mon[31930]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:12.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:12 smithi137 ceph-mon[35868]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:12 smithi193 ceph-mon[36702]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:14 smithi096 ceph-mon[31930]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:14 smithi137 ceph-mon[35868]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:14 smithi193 ceph-mon[36702]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:16.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.151+0000 7fa6c37fe700 1 -- 172.21.15.96:0/3354631131 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6d40fdeb0 con 0x7fa6bc0a2910 2024-03-18T15:36:16.153 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 9 on host 'smithi096' 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.153+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 msgr2=0x7fa6bc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.153+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00a400 tx=0x7fa6cc00d040 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.153+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 msgr2=0x7fa6d419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.153+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400c900 tx=0x7fa6c400cc10 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.153+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 shutdown_connections 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa6bc0a2910 0x7fa6bc0a4dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6d4107fe0 0x7fa6d419fab0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6d4102d80 0x7fa6d41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 --2- 172.21.15.96:0/3354631131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6d41056b0 0x7fa6d4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 >> 172.21.15.96:0/3354631131 conn(0x7fa6d40fc790 msgr2=0x7fa6d41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:36:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 shutdown_connections 2024-03-18T15:36:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:36:16.154+0000 7fa6d8a0f700 1 -- 172.21.15.96:0/3354631131 wait complete. 2024-03-18T15:36:16.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:16 smithi096 ceph-mon[31930]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:16.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:16 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.845 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:16 smithi096 ceph-mon[31930]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T15:36:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:16 smithi137 ceph-mon[35868]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:16 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:16 smithi137 ceph-mon[35868]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T15:36:16.881 DEBUG:teuthology.orchestra.run.smithi096:osd.9> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.9.service 2024-03-18T15:36:16.884 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-03-18T15:36:16.884 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T15:36:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:16 smithi193 ceph-mon[36702]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:16 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:16 smithi193 ceph-mon[36702]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-18T15:36:17.134 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:36:18.147 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:18.148 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:17 smithi096 ceph-mon[31930]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:18.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T15:36:18.278 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T15:36:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:36:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:18.279 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:17 smithi137 ceph-mon[35868]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: osdmap e66: 10 total, 9 up, 10 in 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:18.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:17 smithi193 ceph-mon[36702]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:18.436 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:36:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:36:18.146+0000 7f16d1b2c700 -1 osd.9 0 waiting for initial osdmap 2024-03-18T15:36:18.436 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:36:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:36:18.159+0000 7f16cd91f700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138066 s, 759 MB/s 2024-03-18T15:36:19.191 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-18T15:36:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:19 smithi137 ceph-mon[35868]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:36:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:19 smithi137 ceph-mon[35868]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T15:36:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:19 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.395 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:19 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:19 smithi193 ceph-mon[36702]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:36:19.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:19 smithi193 ceph-mon[36702]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T15:36:19.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:19 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:19 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:19 smithi096 ceph-mon[31930]: from='osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-03-18T15:36:19.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:19 smithi096 ceph-mon[31930]: osdmap e67: 10 total, 9 up, 10 in 2024-03-18T15:36:19.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:19 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:19 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:19.884 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-03-18T15:36:20.124 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] boot 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T15:36:20.402 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.403 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:20 smithi137 ceph-mon[35868]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] boot 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:20 smithi193 ceph-mon[36702]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: osd.9 [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] boot 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: osdmap e68: 10 total, 10 up, 10 in 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-18T15:36:20.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:20 smithi096 ceph-mon[31930]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi096 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi096 to 4272M 2024-03-18T15:36:21.280 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.281 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:21 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi096 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi096 to 4272M 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:21.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:21 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: osdmap e69: 10 total, 10 up, 10 in 2024-03-18T15:36:21.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi096 2024-03-18T15:36:21.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi096 to 4272M 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:21.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:21 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:21.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 -- 172.21.15.137:0/712678148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8102d60 msgr2=0x7f12a8105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:21.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 --2- 172.21.15.137:0/712678148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8102d60 0x7f12a8105150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009990 tx=0x7f12a4009ca0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 -- 172.21.15.137:0/712678148 shutdown_connections 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 --2- 172.21.15.137:0/712678148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 0x7f12a810a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 --2- 172.21.15.137:0/712678148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8102d60 0x7f12a8105150 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 --2- 172.21.15.137:0/712678148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8105690 0x7f12a8107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 -- 172.21.15.137:0/712678148 >> 172.21.15.137:0/712678148 conn(0x7f12a80fc770 msgr2=0x7f12a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 -- 172.21.15.137:0/712678148 shutdown_connections 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.810+0000 7f12b06c9700 1 -- 172.21.15.137:0/712678148 wait complete. 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 Processor -- start 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 -- start start 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8102d60 0x7f12a81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 0x7f12a819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12a810dea0 con 0x7f12a8107fc0 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12a810dd20 con 0x7f12a8105690 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12b06c9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12a810e020 con 0x7f12a8102d60 2024-03-18T15:36:21.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58404/0 (socket says 172.21.15.137:58404) 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 -- 172.21.15.137:0/1901154933 learned_addr learned my addr 172.21.15.137:0/1901154933 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12ae465700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8102d60 0x7f12a81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12aec66700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 0x7f12a819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 -- 172.21.15.137:0/1901154933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8102d60 msgr2=0x7f12a81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8102d60 0x7f12a81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 -- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 msgr2=0x7f12a819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 0x7f12a819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.811+0000 7f12adc64700 1 -- 172.21.15.137:0/1901154933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a81a0110 con 0x7f12a8105690 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f12adc64700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f129800c9b0 tx=0x7f129800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298007b00 con 0x7f12a8105690 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298007ca0 con 0x7f12a8105690 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a81a0400 con 0x7f12a8105690 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12980186f0 con 0x7f12a8105690 2024-03-18T15:36:21.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.812+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a81a0940 con 0x7f12a8105690 2024-03-18T15:36:21.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.813+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a8068f90 con 0x7f12a8105690 2024-03-18T15:36:21.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.815+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1298018890 con 0x7f12a8105690 2024-03-18T15:36:21.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.815+0000 7f129f7fe700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 0x7f12940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:21.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.815+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f12980ce4d0 con 0x7f12a8105690 2024-03-18T15:36:21.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.815+0000 7f12ae465700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 0x7f12940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:21.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.816+0000 7f12ae465700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 0x7f12940a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12a400b5c0 tx=0x7f12a4003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:36:21.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:21.819+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f1298014030 con 0x7f12a8105690 2024-03-18T15:36:22.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:22.022+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f12a8062710 con 0x7f12940a2890 2024-03-18T15:36:22.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:22 smithi137 ceph-mon[35868]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:22.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:22.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:22.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:22.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:22 smithi193 ceph-mon[36702]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:22.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:22.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:22.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:22 smithi096 ceph-mon[31930]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:22.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:23.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:23 smithi193 ceph-mon[36702]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:23.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:23 smithi096 ceph-mon[31930]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:23.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:23 smithi137 ceph-mon[35868]: from='client.24308 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:24.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:24 smithi137 ceph-mon[35868]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:24.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:24 smithi096 ceph-mon[31930]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:24.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:24 smithi193 ceph-mon[36702]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:25.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:25 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/3379549649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:25 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:25 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]': finished 2024-03-18T15:36:25.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:25 smithi096 ceph-mon[31930]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T15:36:25.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:25.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:25 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/3379549649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:25 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:25 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]': finished 2024-03-18T15:36:25.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:25 smithi137 ceph-mon[35868]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T15:36:25.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:25.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:25 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/3379549649' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:25 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]: dispatch 2024-03-18T15:36:25.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:25 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d8d5301-49b5-4393-beb1-a3e682d035c9"}]': finished 2024-03-18T15:36:25.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:25 smithi193 ceph-mon[36702]: osdmap e70: 11 total, 10 up, 11 in 2024-03-18T15:36:25.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:26 smithi096 ceph-mon[31930]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:26 smithi096 ceph-mon[31930]: from='client.? 172.21.15.137:0/2390228227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:26.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:26 smithi137 ceph-mon[35868]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:26.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:26 smithi137 ceph-mon[35868]: from='client.? 172.21.15.137:0/2390228227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:26.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:26 smithi193 ceph-mon[36702]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:26.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:26 smithi193 ceph-mon[36702]: from='client.? 172.21.15.137:0/2390228227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:28.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:28 smithi137 ceph-mon[35868]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:28 smithi193 ceph-mon[36702]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:28 smithi096 ceph-mon[31930]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:30 smithi137 ceph-mon[35868]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:30.743 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:30 smithi193 ceph-mon[36702]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:30 smithi096 ceph-mon[31930]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:32 smithi137 ceph-mon[35868]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:32 smithi193 ceph-mon[36702]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:32 smithi096 ceph-mon[31930]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:33.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:33 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T15:36:33.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:33 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:33.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:33 smithi137 ceph-mon[35868]: Deploying daemon osd.10 on smithi137 2024-03-18T15:36:33.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:33 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T15:36:33.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:33 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:33.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:33 smithi193 ceph-mon[36702]: Deploying daemon osd.10 on smithi137 2024-03-18T15:36:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-18T15:36:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:33 smithi096 ceph-mon[31930]: Deploying daemon osd.10 on smithi137 2024-03-18T15:36:34.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:34 smithi137 ceph-mon[35868]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:34 smithi193 ceph-mon[36702]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:34 smithi096 ceph-mon[31930]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:36.661 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:36 smithi137 ceph-mon[35868]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:36 smithi193 ceph-mon[36702]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:36 smithi096 ceph-mon[31930]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:38.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:38 smithi137 ceph-mon[35868]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:38.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:38 smithi193 ceph-mon[36702]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:38 smithi096 ceph-mon[31930]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:40.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:40 smithi137 ceph-mon[35868]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:40 smithi193 ceph-mon[36702]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:40 smithi096 ceph-mon[31930]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:42.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.344+0000 7f129f7fe700 1 -- 172.21.15.137:0/1901154933 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f12a8062710 con 0x7f12940a2890 2024-03-18T15:36:42.345 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.346+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 msgr2=0x7f12940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.346+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 0x7f12940a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f12a400b5c0 tx=0x7f12a4003960 comp rx=0 tx=0).stop 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.346+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 msgr2=0x7f12a81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.346+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f129800c9b0 tx=0x7f129800ccc0 comp rx=0 tx=0).stop 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 shutdown_connections 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12940a2890 0x7f12940a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12a8107fc0 0x7f12a819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:42.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12a8102d60 0x7f12a81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:42.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 --2- 172.21.15.137:0/1901154933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12a8105690 0x7f12a81989f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:42.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 >> 172.21.15.137:0/1901154933 conn(0x7f12a80fc770 msgr2=0x7f12a81036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:36:42.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 shutdown_connections 2024-03-18T15:36:42.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-18T15:36:42.347+0000 7f12b06c9700 1 -- 172.21.15.137:0/1901154933 wait complete. 2024-03-18T15:36:42.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:42.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:42 smithi137 ceph-mon[35868]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:42.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:42 smithi193 ceph-mon[36702]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:42 smithi096 ceph-mon[31930]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-18T15:36:43.023 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.10.service 2024-03-18T15:36:43.026 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-03-18T15:36:43.026 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-18T15:36:43.273 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-mon[35868]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:44.606 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:36:44.329+0000 7fed6bd84700 -1 osd.10 0 waiting for initial osdmap 2024-03-18T15:36:44.606 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:36:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:36:44.345+0000 7fed67b77700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:44 smithi193 ceph-mon[36702]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: osdmap e71: 11 total, 10 up, 11 in 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: from='osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-18T15:36:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:44 smithi096 ceph-mon[31930]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-18T15:36:45.480 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-18T15:36:45.481 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-18T15:36:45.481 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-03-18T15:36:45.481 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-03-18T15:36:45.481 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113692 s, 922 MB/s 2024-03-18T15:36:45.481 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-03-18T15:36:45.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:45 smithi137 ceph-mon[35868]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:36:45.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:45 smithi137 ceph-mon[35868]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T15:36:45.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:45 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:45.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:45 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:45.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:45 smithi193 ceph-mon[36702]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:36:45.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:45 smithi193 ceph-mon[36702]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T15:36:45.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:45.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:45.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:45 smithi096 ceph-mon[31930]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-18T15:36:45.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:45 smithi096 ceph-mon[31930]: osdmap e72: 11 total, 10 up, 11 in 2024-03-18T15:36:45.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:45 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:45.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:45 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:46.276 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-03-18T15:36:46.519 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] boot 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:46.607 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:46 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] boot 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.641 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:46.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:46.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:46 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:36:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:36:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: osd.10 [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] boot 2024-03-18T15:36:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: osdmap e73: 11 total, 11 up, 11 in 2024-03-18T15:36:46.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: osdmap e74: 11 total, 11 up, 11 in 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:36:46.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:46 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:36:47.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:47 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi137 2024-03-18T15:36:47.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:47 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-18T15:36:47.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:47 smithi137 ceph-mon[35868]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T15:36:47.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:47 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi137 2024-03-18T15:36:47.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:47 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-18T15:36:47.659 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:47 smithi193 ceph-mon[36702]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T15:36:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:47 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi137 2024-03-18T15:36:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:47 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi137 to 4270M 2024-03-18T15:36:47.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:47 smithi096 ceph-mon[31930]: osdmap e75: 11 total, 11 up, 11 in 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 -- 172.21.15.193:0/4262434930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698103ea0 msgr2=0x7fc69810a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 --2- 172.21.15.193:0/4262434930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698103ea0 0x7fc69810a750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc69000b0a0 tx=0x7fc69000b3b0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 -- 172.21.15.193:0/4262434930 shutdown_connections 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 --2- 172.21.15.193:0/4262434930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698103ea0 0x7fc69810a750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 --2- 172.21.15.193:0/4262434930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698102300 0x7fc698102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 --2- 172.21.15.193:0/4262434930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103500 0x7fc698103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 -- 172.21.15.193:0/4262434930 >> 172.21.15.193:0/4262434930 conn(0x7fc6980fd8e0 msgr2=0x7fc6980ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:36:48.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 -- 172.21.15.193:0/4262434930 shutdown_connections 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.235+0000 7fc69e035700 1 -- 172.21.15.193:0/4262434930 wait complete. 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 Processor -- start 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 -- start start 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698103500 0x7fc6981989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103ea0 0x7fc69819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810dee0 con 0x7fc698103ea0 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810dd60 con 0x7fc698103500 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810e060 con 0x7fc698102300 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:49666/0 (socket says 172.21.15.193:49666) 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 -- 172.21.15.193:0/1230876052 learned_addr learned my addr 172.21.15.193:0/1230876052 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc68ffff700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698103500 0x7fc6981989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc697fff700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103ea0 0x7fc69819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:48.247 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 -- 172.21.15.193:0/1230876052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698103500 msgr2=0x7fc6981989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698103500 0x7fc6981989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 -- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103ea0 msgr2=0x7fc69819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103ea0 0x7fc69819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 -- 172.21.15.193:0/1230876052 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6981a0100 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6977fe700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d9b0 tx=0x7fc68800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc688017070 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc688009e10 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6981a03f0 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.236+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68800f620 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.237+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a08e0 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.237+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc698068f90 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.238+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc688009930 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.239+0000 7fc6957fa700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 0x7fc6840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.239+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fc6880cdf90 con 0x7fc698102300 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.239+0000 7fc68ffff700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 0x7fc6840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.240+0000 7fc68ffff700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 0x7fc6840a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc680009c60 tx=0x7fc680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:36:48.248 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.243+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc68809b660 con 0x7fc698102300 2024-03-18T15:36:48.446 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:36:48.445+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc698062710 con 0x7fc6840a2890 2024-03-18T15:36:48.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:48 smithi137 ceph-mon[35868]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:48.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:48 smithi137 ceph-mon[35868]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T15:36:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:48 smithi193 ceph-mon[36702]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:48 smithi193 ceph-mon[36702]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T15:36:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:48 smithi096 ceph-mon[31930]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:48 smithi096 ceph-mon[31930]: osdmap e76: 11 total, 11 up, 11 in 2024-03-18T15:36:49.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:49 smithi137 ceph-mon[35868]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:49.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:49 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:49.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:49 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:49.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:49 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:49.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:49 smithi137 ceph-mon[35868]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T15:36:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:49 smithi193 ceph-mon[36702]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:49 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:49 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:49 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:49 smithi193 ceph-mon[36702]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T15:36:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:49 smithi096 ceph-mon[31930]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:36:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:49 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-18T15:36:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:49 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-18T15:36:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:49 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:49.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:49 smithi096 ceph-mon[31930]: osdmap e77: 11 total, 11 up, 11 in 2024-03-18T15:36:50.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:50 smithi193 ceph-mon[36702]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:36:50.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:50 smithi137 ceph-mon[35868]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:36:50.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:50 smithi096 ceph-mon[31930]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-18T15:36:51.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:51 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/2048820574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:51 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:51 smithi193 ceph-mon[36702]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]': finished 2024-03-18T15:36:51.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:51 smithi193 ceph-mon[36702]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T15:36:51.597 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:51 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:36:51.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:51 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/2048820574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:51 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:51 smithi137 ceph-mon[35868]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]': finished 2024-03-18T15:36:51.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:51 smithi137 ceph-mon[35868]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T15:36:51.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:51 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:36:51.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:51 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/2048820574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:51 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]: dispatch 2024-03-18T15:36:51.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:51 smithi096 ceph-mon[31930]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "09d05136-56ba-4ef6-be49-85d156e3c45a"}]': finished 2024-03-18T15:36:51.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:51 smithi096 ceph-mon[31930]: osdmap e78: 12 total, 11 up, 12 in 2024-03-18T15:36:51.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:51 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:36:52.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:52 smithi137 ceph-mon[35868]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:36:52.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:52 smithi137 ceph-mon[35868]: from='client.? 172.21.15.193:0/684545007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:52.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:52 smithi193 ceph-mon[36702]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:36:52.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:52 smithi193 ceph-mon[36702]: from='client.? 172.21.15.193:0/684545007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:52.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:52 smithi096 ceph-mon[31930]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-03-18T15:36:52.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:52 smithi096 ceph-mon[31930]: from='client.? 172.21.15.193:0/684545007' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-18T15:36:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:54 smithi137 ceph-mon[35868]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:54.878 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:54 smithi193 ceph-mon[36702]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:54 smithi096 ceph-mon[31930]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-18T15:36:56.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:56 smithi193 ceph-mon[36702]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:56.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:56 smithi137 ceph-mon[35868]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:56.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:56 smithi096 ceph-mon[31930]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-03-18T15:36:58.594 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:58 smithi193 ceph-mon[36702]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:58 smithi137 ceph-mon[35868]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:58 smithi096 ceph-mon[31930]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:36:59.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T15:36:59.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:36:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T15:36:59.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:36:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:36:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-18T15:36:59.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:36:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:00.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:00 smithi193 ceph-mon[36702]: Deploying daemon osd.11 on smithi193 2024-03-18T15:37:00.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:00 smithi193 ceph-mon[36702]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:00 smithi137 ceph-mon[35868]: Deploying daemon osd.11 on smithi193 2024-03-18T15:37:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:00 smithi137 ceph-mon[35868]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:00 smithi096 ceph-mon[31930]: Deploying daemon osd.11 on smithi193 2024-03-18T15:37:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:00 smithi096 ceph-mon[31930]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:02.591 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:02 smithi193 ceph-mon[36702]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:02 smithi137 ceph-mon[35868]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:02 smithi096 ceph-mon[31930]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:04 smithi193 ceph-mon[36702]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:04 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:04 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:04.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:04 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:04 smithi137 ceph-mon[35868]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:04 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:04 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:04.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:04 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:04 smithi096 ceph-mon[31930]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:04 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:06.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:06 smithi193 ceph-mon[36702]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:06 smithi137 ceph-mon[35868]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:06 smithi096 ceph-mon[31930]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:08.064 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-03-18T15:37:08.064 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.063+0000 7fc6957fa700 1 -- 172.21.15.193:0/1230876052 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc698062710 con 0x7fc6840a2890 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.065+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 msgr2=0x7fc6840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.065+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 0x7fc6840a4d50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc680009c60 tx=0x7fc680009260 comp rx=0 tx=0).stop 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 msgr2=0x7fc6981984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc68800d9b0 tx=0x7fc68800dcc0 comp rx=0 tx=0).stop 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 shutdown_connections 2024-03-18T15:37:08.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6840a2890 0x7fc6840a4d50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc698103ea0 0x7fc69819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc698102300 0x7fc6981984a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 --2- 172.21.15.193:0/1230876052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc698103500 0x7fc6981989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 >> 172.21.15.193:0/1230876052 conn(0x7fc6980fd8e0 msgr2=0x7fc6981067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 shutdown_connections 2024-03-18T15:37:08.067 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:08.066+0000 7fc69e035700 1 -- 172.21.15.193:0/1230876052 wait complete. 2024-03-18T15:37:08.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:08 smithi193 ceph-mon[36702]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:08.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:08 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:08 smithi193 ceph-mon[36702]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:08 smithi193 ceph-mon[36702]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.739 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.11.service 2024-03-18T15:37:08.742 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-18T15:37:08.742 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T15:37:08.772 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:08 smithi096 ceph-mon[31930]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:08.772 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.772 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:08 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.772 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:08 smithi096 ceph-mon[31930]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.772 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:08 smithi096 ceph-mon[31930]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:08 smithi137 ceph-mon[35868]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:08 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:08 smithi137 ceph-mon[35868]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:08 smithi137 ceph-mon[35868]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-18T15:37:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:09 smithi193 ceph-mon[36702]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:09 smithi096 ceph-mon[31930]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:10.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: osdmap e79: 12 total, 11 up, 12 in 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-03-18T15:37:10.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:09 smithi137 ceph-mon[35868]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 -- 172.21.15.96:0/3181989728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102e10 msgr2=0x7f41bc1096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 --2- 172.21.15.96:0/3181989728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102e10 0x7f41bc1096c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41b400b0d0 tx=0x7f41b400b3e0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 -- 172.21.15.96:0/3181989728 shutdown_connections 2024-03-18T15:37:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 --2- 172.21.15.96:0/3181989728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102e10 0x7f41bc1096c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 --2- 172.21.15.96:0/3181989728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc101270 0x7f41bc101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 --2- 172.21.15.96:0/3181989728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc102470 0x7f41bc1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 -- 172.21.15.96:0/3181989728 >> 172.21.15.96:0/3181989728 conn(0x7f41bc0fc830 msgr2=0x7f41bc0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 -- 172.21.15.96:0/3181989728 shutdown_connections 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.719+0000 7f41c3028700 1 -- 172.21.15.96:0/3181989728 wait complete. 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 Processor -- start 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 -- start start 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc101270 0x7f41bc105350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc102e10 0x7f41bc103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc10cee0 con 0x7f41bc102470 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc10cd60 con 0x7f41bc101270 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c3028700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc10d060 con 0x7f41bc102e10 2024-03-18T15:37:10.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41bbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41bbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51956/0 (socket says 172.21.15.96:51956) 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41bbfff700 1 -- 172.21.15.96:0/671402848 learned_addr learned my addr 172.21.15.96:0/671402848 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c0dc4700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc101270 0x7f41bc105350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.720+0000 7f41c15c5700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc102e10 0x7f41bc103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 -- 172.21.15.96:0/671402848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc102e10 msgr2=0x7f41bc103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc102e10 0x7f41bc103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 -- 172.21.15.96:0/671402848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc101270 msgr2=0x7f41bc105350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc101270 0x7f41bc105350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 -- 172.21.15.96:0/671402848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc104790 con 0x7f41bc102470 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41bbfff700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b530 tx=0x7f41a800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015020 con 0x7f41bc102470 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc1a27d0 con 0x7f41bc102470 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8011d10 con 0x7f41bc102470 2024-03-18T15:37:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a800f2e0 con 0x7f41bc102470 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.721+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc1a2d90 con 0x7f41bc102470 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.722+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc068f90 con 0x7f41bc102470 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.723+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f41a8018020 con 0x7f41bc102470 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.724+0000 7f41b9ffb700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 0x7f41ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.724+0000 7f41c0dc4700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 0x7f41ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.724+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f41a80cdb80 con 0x7f41bc102470 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.724+0000 7f41c0dc4700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 0x7f41ac0a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f41bc1022d0 tx=0x7f41b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:10.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.728+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f41a801f020 con 0x7f41bc102470 2024-03-18T15:37:10.789 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:37:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:37:10.433+0000 7fb848d3c700 -1 osd.11 0 waiting for initial osdmap 2024-03-18T15:37:10.789 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:37:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:37:10.446+0000 7fb840b27700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:37:10.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.924+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f41bc1a3680 con 0x7f41bc102470 2024-03-18T15:37:10.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.925+0000 7f41b9ffb700 1 -- 172.21.15.96:0/671402848 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f41a809b1e0 con 0x7f41bc102470 2024-03-18T15:37:10.926 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:10.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 msgr2=0x7f41ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:10.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 0x7f41ac0a4d50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f41bc1022d0 tx=0x7f41b0009190 comp rx=0 tx=0).stop 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 msgr2=0x7f41bc1039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f41a800b530 tx=0x7f41a800b840 comp rx=0 tx=0).stop 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 shutdown_connections 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41ac0a2890 0x7f41ac0a4d50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41bc102e10 0x7f41bc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41bc101270 0x7f41bc105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 --2- 172.21.15.96:0/671402848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41bc102470 0x7f41bc1039a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:10.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 >> 172.21.15.96:0/671402848 conn(0x7f41bc0fc830 msgr2=0x7f41bc06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:10.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 shutdown_connections 2024-03-18T15:37:10.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:10.927+0000 7f41c3028700 1 -- 172.21.15.96:0/671402848 wait complete. 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: from='osd.11 ' entity='osd.11' 2024-03-18T15:37:11.100 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:11 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/671402848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: from='osd.11 ' entity='osd.11' 2024-03-18T15:37:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:11 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/671402848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:11.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-03-18T15:37:11.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: osdmap e80: 12 total, 11 up, 12 in 2024-03-18T15:37:11.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:11.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: from='osd.11 ' entity='osd.11' 2024-03-18T15:37:11.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:11 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/671402848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:11.680 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1710776205,"num_in_osds":12,"osd_in_since":1710776211,"num_remapped_pgs":0} 2024-03-18T15:37:12.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: purged_snaps scrub starts 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: purged_snaps scrub ok 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] boot 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T15:37:12.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:12 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: purged_snaps scrub starts 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: purged_snaps scrub ok 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] boot 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T15:37:12.386 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:12 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: purged_snaps scrub starts 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: purged_snaps scrub ok 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: osd.11 [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] boot 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: osdmap e81: 12 total, 12 up, 12 in 2024-03-18T15:37:12.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:12 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-18T15:37:12.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-18T15:37:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:37:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: Detected new or changed devices on smithi193 2024-03-18T15:37:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi193 to 4270M 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:13 smithi096 ceph-mon[31930]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: Detected new or changed devices on smithi193 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi193 to 4270M 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:13 smithi137 ceph-mon[35868]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T15:37:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: Detected new or changed devices on smithi193 2024-03-18T15:37:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi193 to 4270M 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:13.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:13.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:13 smithi193 ceph-mon[36702]: osdmap e82: 12 total, 12 up, 12 in 2024-03-18T15:37:14.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.532+0000 7f15666ac700 1 -- 172.21.15.96:0/1250608007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101f10 msgr2=0x7f1560102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.532+0000 7f15666ac700 1 --2- 172.21.15.96:0/1250608007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101f10 0x7f1560102390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f155800b0a0 tx=0x7f155800b3b0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- 172.21.15.96:0/1250608007 shutdown_connections 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- 172.21.15.96:0/1250608007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101f10 0x7f1560102390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- 172.21.15.96:0/1250608007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560100bf0 0x7f1560101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- 172.21.15.96:0/1250608007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560101550 0x7f15601019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- 172.21.15.96:0/1250608007 >> 172.21.15.96:0/1250608007 conn(0x7f15600fc790 msgr2=0x7f15600febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- 172.21.15.96:0/1250608007 shutdown_connections 2024-03-18T15:37:14.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- 172.21.15.96:0/1250608007 wait complete. 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 Processor -- start 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- start start 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100bf0 0x7f156019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560101f10 0x7f15601a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:14.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.533+0000 7f15666ac700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15601100b0 con 0x7f1560101550 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f15666ac700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f156010ff30 con 0x7f1560100bf0 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f15666ac700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1560110230 con 0x7f1560101f10 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54714/0 (socket says 172.21.15.96:54714) 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 -- 172.21.15.96:0/3200714566 learned_addr learned my addr 172.21.15.96:0/3200714566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155ffff700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100bf0 0x7f156019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f1564c49700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560101f10 0x7f15601a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 -- 172.21.15.96:0/3200714566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560101f10 msgr2=0x7f15601a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560101f10 0x7f15601a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 -- 172.21.15.96:0/3200714566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100bf0 msgr2=0x7f156019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100bf0 0x7f156019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 -- 172.21.15.96:0/3200714566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15601a2290 con 0x7f1560101550 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155f7fe700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f155400d980 tx=0x7f155400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:14.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554016070 con 0x7f1560101550 2024-03-18T15:37:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15601a2580 con 0x7f1560101550 2024-03-18T15:37:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f155400de70 con 0x7f1560101550 2024-03-18T15:37:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155400f3c0 con 0x7f1560101550 2024-03-18T15:37:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.534+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15601a2ac0 con 0x7f1560101550 2024-03-18T15:37:14.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.535+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1560068f90 con 0x7f1560101550 2024-03-18T15:37:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.536+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f1554019070 con 0x7f1560101550 2024-03-18T15:37:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.537+0000 7f155d7fa700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 0x7f15480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.537+0000 7f155ffff700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 0x7f15480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.537+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f15540ce0f0 con 0x7f1560101550 2024-03-18T15:37:14.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.537+0000 7f155ffff700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 0x7f15480a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1550005950 tx=0x7f155000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:14.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.541+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f155409b630 con 0x7f1560101550 2024-03-18T15:37:14.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:14 smithi096 ceph-mon[31930]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.747+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1560062ee0 con 0x7f1560101550 2024-03-18T15:37:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.747+0000 7f155d7fa700 1 -- 172.21.15.96:0/3200714566 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f155409af30 con 0x7f1560101550 2024-03-18T15:37:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:14.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 msgr2=0x7f15480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:14.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 0x7f15480a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1550005950 tx=0x7f155000b410 comp rx=0 tx=0).stop 2024-03-18T15:37:14.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 msgr2=0x7f156019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f155400d980 tx=0x7f155400dc90 comp rx=0 tx=0).stop 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 shutdown_connections 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f15480a2890 0x7f15480a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1560101f10 0x7f15601a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1560100bf0 0x7f156019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 --2- 172.21.15.96:0/3200714566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1560101550 0x7f156019ab70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 >> 172.21.15.96:0/3200714566 conn(0x7f15600fc790 msgr2=0x7f15600fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.749+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 shutdown_connections 2024-03-18T15:37:14.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:14.750+0000 7f15666ac700 1 -- 172.21.15.96:0/3200714566 wait complete. 2024-03-18T15:37:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:14 smithi137 ceph-mon[35868]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:14 smithi193 ceph-mon[36702]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:15.500 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1710776231,"num_in_osds":12,"osd_in_since":1710776211,"num_remapped_pgs":0} 2024-03-18T15:37:15.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T15:37:15.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:15 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3200714566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:15.740 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:37:15.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:15 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3200714566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:15 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3200714566' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-18T15:37:16.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:16 smithi096 ceph-mon[31930]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:16 smithi137 ceph-mon[35868]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:16 smithi193 ceph-mon[36702]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.402+0000 7fd4c4b51700 1 -- 172.21.15.96:0/4244232597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 msgr2=0x7fd4c01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.402+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/4244232597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c01027a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40099c0 tx=0x7fd4b4009cd0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 -- 172.21.15.96:0/4244232597 shutdown_connections 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/4244232597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 0x7fd4c010f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/4244232597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 0x7fd4c01086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/4244232597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c01027a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 -- 172.21.15.96:0/4244232597 >> 172.21.15.96:0/4244232597 conn(0x7fd4c0079400 msgr2=0x7fd4c0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 -- 172.21.15.96:0/4244232597 shutdown_connections 2024-03-18T15:37:17.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 -- 172.21.15.96:0/4244232597 wait complete. 2024-03-18T15:37:17.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 Processor -- start 2024-03-18T15:37:17.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.403+0000 7fd4c4b51700 1 -- start start 2024-03-18T15:37:17.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:17.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 0x7fd4c0074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:17.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 0x7fd4c0075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:17.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0112600 con 0x7fd4c0102320 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0112480 con 0x7fd4c0108300 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0112780 con 0x7fd4c0102ce0 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54734/0 (socket says 172.21.15.96:54734) 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 -- 172.21.15.96:0/1321108469 learned_addr learned my addr 172.21.15.96:0/1321108469 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4bdd9b700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 0x7fd4c0074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4bed9d700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 0x7fd4c0075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 -- 172.21.15.96:0/1321108469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 msgr2=0x7fd4c0074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 0x7fd4c0074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 -- 172.21.15.96:0/1321108469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 msgr2=0x7fd4c0075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 0x7fd4c0075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 -- 172.21.15.96:0/1321108469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c01a6f50 con 0x7fd4c0102320 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4be59c700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d950 tx=0x7fd4b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:17.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0016070 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c01a7240 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b0009e20 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f620 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.404+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c01a77a0 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.405+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c0068f90 con 0x7fd4c0102320 2024-03-18T15:37:17.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.407+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd4b0019070 con 0x7fd4c0102320 2024-03-18T15:37:17.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.407+0000 7fd4af7fe700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 0x7fd4a80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:17.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.407+0000 7fd4bdd9b700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 0x7fd4a80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:17.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.407+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd4b00ce1d0 con 0x7fd4c0102320 2024-03-18T15:37:17.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.408+0000 7fd4bdd9b700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 0x7fd4a80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0076010 tx=0x7fd4b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:17.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.411+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd4b009b710 con 0x7fd4c0102320 2024-03-18T15:37:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.609+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4c002db70 con 0x7fd4c0102320 2024-03-18T15:37:17.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.610+0000 7fd4af7fe700 1 -- 172.21.15.96:0/1321108469 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18408 (secure 0 0 0) 0x7fd4b009b010 con 0x7fd4c0102320 2024-03-18T15:37:17.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:17.611 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":82,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:37:13.033553+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 msgr2=0x7fd4a80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 0x7fd4a80a4d50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0076010 tx=0x7fd4b4005fb0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 msgr2=0x7fd4c00745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d950 tx=0x7fd4b000dc60 comp rx=0 tx=0).stop 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 shutdown_connections 2024-03-18T15:37:17.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4a80a2890 0x7fd4a80a4d50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4c0108300 0x7fd4c0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4c0102320 0x7fd4c00745d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 --2- 172.21.15.96:0/1321108469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4c0102ce0 0x7fd4c0074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 >> 172.21.15.96:0/1321108469 conn(0x7fd4c0079400 msgr2=0x7fd4c0100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 shutdown_connections 2024-03-18T15:37:17.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:17.612+0000 7fd4c4b51700 1 -- 172.21.15.96:0/1321108469 wait complete. 2024-03-18T15:37:18.320 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:37:18.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-18T15:37:18.557 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:37:18.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:18 smithi096 ceph-mon[31930]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:18.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:18 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1321108469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:18 smithi137 ceph-mon[35868]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:18 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1321108469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:18 smithi193 ceph-mon[36702]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:18 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1321108469' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:20.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 -- 172.21.15.96:0/725796146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe8101bc0 msgr2=0x7f4fe810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:20.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 --2- 172.21.15.96:0/725796146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe8101bc0 0x7f4fe810e000 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000b0d0 tx=0x7f4fe000b3e0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 -- 172.21.15.96:0/725796146 shutdown_connections 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 --2- 172.21.15.96:0/725796146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe8101bc0 0x7f4fe810e000 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 --2- 172.21.15.96:0/725796146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe81071e0 0x7f4fe81075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 --2- 172.21.15.96:0/725796146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 0x7f4fe8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 -- 172.21.15.96:0/725796146 >> 172.21.15.96:0/725796146 conn(0x7f4fe80771c0 msgr2=0x7f4fe80775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 -- 172.21.15.96:0/725796146 shutdown_connections 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.290+0000 7f4fed363700 1 -- 172.21.15.96:0/725796146 wait complete. 2024-03-18T15:37:20.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 Processor -- start 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 -- start start 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 0x7f4fe8102e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe8101bc0 0x7f4fe8103390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe81114e0 con 0x7f4fe81071e0 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8111360 con 0x7f4fe8101200 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fed363700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe8111660 con 0x7f4fe8101bc0 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:20.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fe77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54752/0 (socket says 172.21.15.96:54752) 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.291+0000 7f4fe77fe700 1 -- 172.21.15.96:0/2507764545 learned_addr learned my addr 172.21.15.96:0/2507764545 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe67fc700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe8101bc0 0x7f4fe8103390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe6ffd700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 0x7f4fe8102e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 -- 172.21.15.96:0/2507764545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe8101bc0 msgr2=0x7f4fe8103390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe8101bc0 0x7f4fe8103390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 -- 172.21.15.96:0/2507764545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 msgr2=0x7f4fe8102e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 0x7f4fe8102e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 -- 172.21.15.96:0/2507764545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe81a6e90 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fe77fe700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006070 tx=0x7f4fe0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0021070 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe0007c00 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe81a7120 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00052a0 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.292+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe81a75d0 con 0x7f4fe81071e0 2024-03-18T15:37:20.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.293+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe8068f90 con 0x7f4fe81071e0 2024-03-18T15:37:20.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.294+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f4fe0024070 con 0x7f4fe81071e0 2024-03-18T15:37:20.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.295+0000 7f4fcffff700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 0x7f4fd00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:20.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.295+0000 7f4fe6ffd700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 0x7f4fd00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:20.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.295+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4fe000f020 con 0x7f4fe81071e0 2024-03-18T15:37:20.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.295+0000 7f4fe6ffd700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 0x7f4fd00a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009a10 tx=0x7f4fdc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:20.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.298+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4fe002b020 con 0x7f4fe81071e0 2024-03-18T15:37:20.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.494+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fe81a78d0 con 0x7f4fe81071e0 2024-03-18T15:37:20.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.495+0000 7f4fcffff700 1 -- 172.21.15.96:0/2507764545 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f4fe0007db0 con 0x7f4fe81071e0 2024-03-18T15:37:20.496 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 msgr2=0x7f4fd00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 0x7f4fd00a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009a10 tx=0x7f4fdc009190 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 msgr2=0x7f4fe81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006070 tx=0x7f4fe0004e40 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 shutdown_connections 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4fd00a2890 0x7f4fd00a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fe81071e0 0x7f4fe81038d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fe8101200 0x7f4fe8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 --2- 172.21.15.96:0/2507764545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fe8101bc0 0x7f4fe8103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 >> 172.21.15.96:0/2507764545 conn(0x7f4fe80771c0 msgr2=0x7f4fe81003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 shutdown_connections 2024-03-18T15:37:20.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:20.497+0000 7f4fed363700 1 -- 172.21.15.96:0/2507764545 wait complete. 2024-03-18T15:37:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:20 smithi137 ceph-mon[35868]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:20.870 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:20 smithi096 ceph-mon[31930]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:20 smithi193 ceph-mon[36702]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:21.238 INFO:tasks.cephadm:Adding mds.a on smithi096 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.d on smithi096 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.g on smithi096 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.j on smithi096 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-03-18T15:37:21.239 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-03-18T15:37:21.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph orch apply mds all '12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-03-18T15:37:21.491 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:37:21.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:21 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2507764545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:21.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:21 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2507764545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:21.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:21 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2507764545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:22.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:22 smithi096 ceph-mon[31930]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:22.811 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:22 smithi193 ceph-mon[36702]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:22.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:22 smithi137 ceph-mon[35868]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.175+0000 7f198864d700 1 -- 172.21.15.193:0/4122803906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980102400 msgr2=0x7f1980102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.175+0000 7f198864d700 1 --2- 172.21.15.193:0/4122803906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980102400 0x7f1980102860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1970009990 tx=0x7f1970009ca0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- 172.21.15.193:0/4122803906 shutdown_connections 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- 172.21.15.193:0/4122803906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102da0 0x7f1980109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- 172.21.15.193:0/4122803906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980101200 0x7f1980101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- 172.21.15.193:0/4122803906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980102400 0x7f1980102860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- 172.21.15.193:0/4122803906 >> 172.21.15.193:0/4122803906 conn(0x7f19800fc7a0 msgr2=0x7f19800febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:23.178 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- 172.21.15.193:0/4122803906 shutdown_connections 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- 172.21.15.193:0/4122803906 wait complete. 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 Processor -- start 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- start start 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102400 0x7f198019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980102da0 0x7f19801a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f198010ce00 con 0x7f1980102da0 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f198010cc80 con 0x7f1980102400 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f198864d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f198010cf80 con 0x7f1980101200 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f19863e9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f19863e9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:53486/0 (socket says 172.21.15.193:53486) 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.176+0000 7f19863e9700 1 -- 172.21.15.193:0/3592327004 learned_addr learned my addr 172.21.15.193:0/3592327004 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f1986bea700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980102da0 0x7f19801a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:23.179 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f1985be8700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102400 0x7f198019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 -- 172.21.15.193:0/3592327004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102400 msgr2=0x7f198019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102400 0x7f198019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 -- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980102da0 msgr2=0x7f19801a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980102da0 0x7f19801a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 -- 172.21.15.193:0/3592327004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a23e0 con 0x7f1980101200 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19863e9700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f197c00e970 tx=0x7f197c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c004220 con 0x7f1980101200 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a26d0 con 0x7f1980101200 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197c0043c0 con 0x7f1980101200 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c003dc0 con 0x7f1980101200 2024-03-18T15:37:23.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.177+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a2c60 con 0x7f1980101200 2024-03-18T15:37:23.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.178+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1980068f90 con 0x7f1980101200 2024-03-18T15:37:23.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.179+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f197c00c7e0 con 0x7f1980101200 2024-03-18T15:37:23.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.180+0000 7f19777fe700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 0x7f196c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:23.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.180+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f197c0ce790 con 0x7f1980101200 2024-03-18T15:37:23.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.180+0000 7f1985be8700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 0x7f196c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:23.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.180+0000 7f1985be8700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 0x7f196c0a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f198006b060 tx=0x7f197000b8f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:23.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.184+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f197c09bcd0 con 0x7f1980101200 2024-03-18T15:37:23.388 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.388+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7f1980062910 con 0x7f196c0a2890 2024-03-18T15:37:23.396 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.396+0000 7f19777fe700 1 -- 172.21.15.193:0/3592327004 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1980062910 con 0x7f196c0a2890 2024-03-18T15:37:23.396 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 msgr2=0x7f196c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 0x7f196c0a4d50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f198006b060 tx=0x7f197000b8f0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 msgr2=0x7f198006aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f197c00e970 tx=0x7f197c00ec80 comp rx=0 tx=0).stop 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 shutdown_connections 2024-03-18T15:37:23.398 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f196c0a2890 0x7f196c0a4d50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1980102da0 0x7f19801a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1980101200 0x7f198006aae0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 --2- 172.21.15.193:0/3592327004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1980102400 0x7f198019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 >> 172.21.15.193:0/3592327004 conn(0x7f19800fc7a0 msgr2=0x7f1980105780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 shutdown_connections 2024-03-18T15:37:23.399 INFO:teuthology.orchestra.run.smithi193.stderr:2024-03-18T15:37:23.398+0000 7f198864d700 1 -- 172.21.15.193:0/3592327004 wait complete. 2024-03-18T15:37:23.988 DEBUG:teuthology.orchestra.run.smithi096:mds.a> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.a.service 2024-03-18T15:37:23.991 DEBUG:teuthology.orchestra.run.smithi096:mds.d> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.d.service 2024-03-18T15:37:24.033 DEBUG:teuthology.orchestra.run.smithi096:mds.g> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.g.service 2024-03-18T15:37:24.076 DEBUG:teuthology.orchestra.run.smithi096:mds.j> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.j.service 2024-03-18T15:37:24.120 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.b.service 2024-03-18T15:37:24.122 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.e.service 2024-03-18T15:37:24.165 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.h.service 2024-03-18T15:37:24.209 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.k.service 2024-03-18T15:37:24.252 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.c.service 2024-03-18T15:37:24.254 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.f.service 2024-03-18T15:37:24.298 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.i.service 2024-03-18T15:37:24.386 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.l.service 2024-03-18T15:37:24.429 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-18T15:37:24.429 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:37:24.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 -- 172.21.15.96:0/4163787553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938100720 msgr2=0x7fe938100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 --2- 172.21.15.96:0/4163787553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938100720 0x7fe938100b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe930009ca0 tx=0x7fe93002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 -- 172.21.15.96:0/4163787553 shutdown_connections 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 --2- 172.21.15.96:0/4163787553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 0x7fe938108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 --2- 172.21.15.96:0/4163787553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938101910 0x7fe938101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 --2- 172.21.15.96:0/4163787553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938100720 0x7fe938100b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 -- 172.21.15.96:0/4163787553 >> 172.21.15.96:0/4163787553 conn(0x7fe9380fbc90 msgr2=0x7fe9380fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 -- 172.21.15.96:0/4163787553 shutdown_connections 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.538+0000 7fe93ec85640 1 -- 172.21.15.96:0/4163787553 wait complete. 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 Processor -- start 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 -- start start 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938100720 0x7fe938197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 0x7fe93819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810d4c0 con 0x7fe938101910 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810d340 con 0x7fe9381022d0 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ec85640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe93810d640 con 0x7fe938100720 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46814/0 (socket says 172.21.15.96:46814) 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 -- 172.21.15.96:0/958956323 learned_addr learned my addr 172.21.15.96:0/958956323 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93ca21640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938100720 0x7fe938197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe93d222640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 0x7fe93819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 -- 172.21.15.96:0/958956323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938100720 msgr2=0x7fe938197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938100720 0x7fe938197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 -- 172.21.15.96:0/958956323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 msgr2=0x7fe93819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 0x7fe93819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 -- 172.21.15.96:0/958956323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93819f770 con 0x7fe938101910 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.539+0000 7fe937fff640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d950 tx=0x7fe92000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.540+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920017030 con 0x7fe938101910 2024-03-18T15:37:24.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.540+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920004020 con 0x7fe938101910 2024-03-18T15:37:24.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.540+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9200047a0 con 0x7fe938101910 2024-03-18T15:37:24.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.540+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93819fa60 con 0x7fe938101910 2024-03-18T15:37:24.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.540+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93819ffb0 con 0x7fe938101910 2024-03-18T15:37:24.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.541+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938107ac0 con 0x7fe938101910 2024-03-18T15:37:24.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.542+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe9200041d0 con 0x7fe938101910 2024-03-18T15:37:24.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.543+0000 7fe935ffb640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 0x7fe9240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.543+0000 7fe93ca21640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 0x7fe9240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.543+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe9200cdf60 con 0x7fe938101910 2024-03-18T15:37:24.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.543+0000 7fe93ca21640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 0x7fe9240a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe93002c700 tx=0x7fe930032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:24.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.547+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe92009b4a0 con 0x7fe938101910 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:24.630 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:24.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:24.631 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:24 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:24.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:24.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:24.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:24 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.724+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe93802d3d0 con 0x7fe938101910 2024-03-18T15:37:24.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.725+0000 7fe935ffb640 1 -- 172.21.15.96:0/958956323 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18408 (secure 0 0 0) 0x7fe92009ada0 con 0x7fe938101910 2024-03-18T15:37:24.726 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:24.727 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":82,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:37:13.033553+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 msgr2=0x7fe9240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 0x7fe9240a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe93002c700 tx=0x7fe930032590 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 msgr2=0x7fe938198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d950 tx=0x7fe92000de10 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 shutdown_connections 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe9240a2800 0x7fe9240a4cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9381022d0 0x7fe93819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe938101910 0x7fe938198090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 --2- 172.21.15.96:0/958956323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe938100720 0x7fe938197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 >> 172.21.15.96:0/958956323 conn(0x7fe9380fbc90 msgr2=0x7fe938104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:24.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 shutdown_connections 2024-03-18T15:37:24.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.728+0000 7fe93ec85640 1 -- 172.21.15.96:0/958956323 wait complete. 2024-03-18T15:37:24.739 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:37:24.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:37:24.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 -- 172.21.15.96:0/3227586738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 msgr2=0x7ff85c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 --2- 172.21.15.96:0/3227586738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c100b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff858009900 tx=0x7ff85802bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:24.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 -- 172.21.15.96:0/3227586738 shutdown_connections 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 --2- 172.21.15.96:0/3227586738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1022b0 0x7ff85c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 --2- 172.21.15.96:0/3227586738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1018f0 0x7ff85c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 --2- 172.21.15.96:0/3227586738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c100b20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 -- 172.21.15.96:0/3227586738 >> 172.21.15.96:0/3227586738 conn(0x7ff85c0fbc90 msgr2=0x7ff85c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 -- 172.21.15.96:0/3227586738 shutdown_connections 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.843+0000 7ff863c75640 1 -- 172.21.15.96:0/3227586738 wait complete. 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 Processor -- start 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 -- start start 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1018f0 0x7ff85c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 0x7ff85c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c370 con 0x7ff85c100700 2024-03-18T15:37:24.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c1f0 con 0x7ff85c1018f0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff863c75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c10c4f0 con 0x7ff85c1022b0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46828/0 (socket says 172.21.15.96:46828) 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 -- 172.21.15.96:0/121300633 learned_addr learned my addr 172.21.15.96:0/121300633 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff862212640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 0x7ff85c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861210640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1018f0 0x7ff85c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 -- 172.21.15.96:0/121300633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 msgr2=0x7ff85c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 0x7ff85c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 -- 172.21.15.96:0/121300633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1018f0 msgr2=0x7ff85c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1018f0 0x7ff85c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff861a11640 1 -- 172.21.15.96:0/121300633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c19f7a0 con 0x7ff85c100700 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.844+0000 7ff862212640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 0x7ff85c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff861a11640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858041070 con 0x7ff85c100700 2024-03-18T15:37:24.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8580054e0 con 0x7ff85c100700 2024-03-18T15:37:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c19fa30 con 0x7ff85c100700 2024-03-18T15:37:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580346a0 con 0x7ff85c100700 2024-03-18T15:37:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.845+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c19ff20 con 0x7ff85c100700 2024-03-18T15:37:24.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.846+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c1a0150 con 0x7ff85c100700 2024-03-18T15:37:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.847+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff858044070 con 0x7ff85c100700 2024-03-18T15:37:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.847+0000 7ff852ffd640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 0x7ff8480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.848+0000 7ff861210640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 0x7ff8480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.848+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8580f03b0 con 0x7ff85c100700 2024-03-18T15:37:24.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.848+0000 7ff861210640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 0x7ff8480a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff85c199090 tx=0x7ff84c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:24.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:24.852+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff8580bd8f0 con 0x7ff85c100700 2024-03-18T15:37:24.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:37:24.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-03-18T15:37:24.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:24 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.027+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff85c1a0320 con 0x7ff85c100700 2024-03-18T15:37:25.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.027+0000 7ff852ffd640 1 -- 172.21.15.96:0/121300633 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7ff8580bd1f0 con 0x7ff85c100700 2024-03-18T15:37:25.028 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:37:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 msgr2=0x7ff8480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 0x7ff8480a4cc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff85c199090 tx=0x7ff84c005c60 comp rx=0 tx=0).stop 2024-03-18T15:37:25.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 msgr2=0x7ff85c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff858000c00 tx=0x7ff858004040 comp rx=0 tx=0).stop 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 shutdown_connections 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8480a2800 0x7ff8480a4cc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff85c1022b0 0x7ff85c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff85c1018f0 0x7ff85c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 --2- 172.21.15.96:0/121300633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff85c100700 0x7ff85c197b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 >> 172.21.15.96:0/121300633 conn(0x7ff85c0fbc90 msgr2=0x7ff85c069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 shutdown_connections 2024-03-18T15:37:25.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.029+0000 7ff863c75640 1 -- 172.21.15.96:0/121300633 wait complete. 2024-03-18T15:37:25.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:25.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 -- 172.21.15.96:0/4021760954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c04101800 msgr2=0x7f5c04101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/4021760954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c04101800 0x7f5c04101c20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009900 tx=0x7f5c0002bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:25.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 -- 172.21.15.96:0/4021760954 shutdown_connections 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/4021760954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 0x7f5c04109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/4021760954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c041029f0 0x7f5c04102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/4021760954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c04101800 0x7f5c04101c20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 -- 172.21.15.96:0/4021760954 >> 172.21.15.96:0/4021760954 conn(0x7f5c040fcd90 msgr2=0x7f5c040ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 -- 172.21.15.96:0/4021760954 shutdown_connections 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.140+0000 7f5c0c13b640 1 -- 172.21.15.96:0/4021760954 wait complete. 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 Processor -- start 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- start start 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c04101800 0x7f5c04075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 0x7f5c04074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0410d450 con 0x7f5c041029f0 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0410d2d0 con 0x7f5c041033b0 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0410d5d0 con 0x7f5c04101800 2024-03-18T15:37:25.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46848/0 (socket says 172.21.15.96:46848) 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 -- 172.21.15.96:0/893681597 learned_addr learned my addr 172.21.15.96:0/893681597 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c09ed7640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c04101800 0x7f5c04075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0a6d8640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 0x7f5c04074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 -- 172.21.15.96:0/893681597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c04101800 msgr2=0x7f5c04075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c04101800 0x7f5c04075590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 -- 172.21.15.96:0/893681597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 msgr2=0x7f5c04074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 0x7f5c04074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 -- 172.21.15.96:0/893681597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040749a0 con 0x7f5c041029f0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c096d6640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c890 tx=0x7f5bf400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004220 con 0x7f5c041029f0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bf40043c0 con 0x7f5c041029f0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c041a1be0 con 0x7f5c041029f0 2024-03-18T15:37:25.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40049e0 con 0x7f5c041029f0 2024-03-18T15:37:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.141+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a2130 con 0x7f5c041029f0 2024-03-18T15:37:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.142+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c04108b20 con 0x7f5c041029f0 2024-03-18T15:37:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.143+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f5bf40079b0 con 0x7f5c041029f0 2024-03-18T15:37:25.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.144+0000 7f5bfaffd640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 0x7f5bf00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:25.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.144+0000 7f5c09ed7640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 0x7f5bf00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:25.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.144+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5bf40ce710 con 0x7f5c041029f0 2024-03-18T15:37:25.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.145+0000 7f5c09ed7640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 0x7f5bf00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c00000c00 tx=0x7f5c00004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:25.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.148+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5bf4014030 con 0x7f5c041029f0 2024-03-18T15:37:25.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.359+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c0402d1b0 con 0x7f5c041029f0 2024-03-18T15:37:25.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.360+0000 7f5bfaffd640 1 -- 172.21.15.96:0/893681597 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f5bf409bc50 con 0x7f5c041029f0 2024-03-18T15:37:25.361 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:25.361 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-18T15:37:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 1 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 msgr2=0x7f5bf00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 0x7f5bf00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5c00000c00 tx=0x7f5c00004f10 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 msgr2=0x7f5c04073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c890 tx=0x7f5bf400cd50 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 shutdown_connections 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bf00a2800 0x7f5bf00a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c041033b0 0x7f5c04074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c041029f0 0x7f5c04073bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 --2- 172.21.15.96:0/893681597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c04101800 0x7f5c04075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 >> 172.21.15.96:0/893681597 conn(0x7f5c040fcd90 msgr2=0x7f5c04105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 shutdown_connections 2024-03-18T15:37:25.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:25.362+0000 7f5c0c13b640 1 -- 172.21.15.96:0/893681597 wait complete. 2024-03-18T15:37:25.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:25 smithi193 ceph-mon[36702]: Deploying daemon mds.l on smithi193 2024-03-18T15:37:25.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:25 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/958956323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:25.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:25 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/121300633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:25.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:25 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/893681597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:25.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:25 smithi096 ceph-mon[31930]: Deploying daemon mds.l on smithi193 2024-03-18T15:37:25.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:25 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/958956323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:25.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:25 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/121300633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:25.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:25 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/893681597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:25.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:25 smithi137 ceph-mon[35868]: Deploying daemon mds.l on smithi193 2024-03-18T15:37:25.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:25 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/958956323' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:37:25.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:25 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/121300633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:37:25.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:25 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/893681597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:26.388 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 systemd[1]: Starting Ceph mds.l for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:26.660 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:26 smithi193 ceph-mon[36702]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:26.660 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 podman[67061]: 2024-03-18 15:37:26.290983374 +0000 UTC m=+0.017503022 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:26.660 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 podman[67061]: 2024-03-18 15:37:26.422481874 +0000 UTC m=+0.149001503 container create 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T15:37:26.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:26 smithi096 ceph-mon[31930]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:26.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:26 smithi137 ceph-mon[35868]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 podman[67061]: 2024-03-18 15:37:26.66033236 +0000 UTC m=+0.386851998 container init 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 podman[67061]: 2024-03-18 15:37:26.662959529 +0000 UTC m=+0.389479163 container start 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 ceph-mds[67080]: -- 172.21.15.193:0/2651989192 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1c99e9a40 con 0x55f1c99bc800 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: starting mds.l at 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 bash[67061]: 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d 2024-03-18T15:37:26.918 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:37:26 smithi193 systemd[1]: Started Ceph mds.l for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:27.374 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 -- 172.21.15.96:0/4209897125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 msgr2=0x7f2028108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 --2- 172.21.15.96:0/4209897125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f2028108b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2020009ca0 tx=0x7f202002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 -- 172.21.15.96:0/4209897125 shutdown_connections 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 --2- 172.21.15.96:0/4209897125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f2028108b60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 --2- 172.21.15.96:0/4209897125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 0x7f2028101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 --2- 172.21.15.96:0/4209897125 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 0x7f2028100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 -- 172.21.15.96:0/4209897125 >> 172.21.15.96:0/4209897125 conn(0x7f20280fbc90 msgr2=0x7f20280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 -- 172.21.15.96:0/4209897125 shutdown_connections 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 -- 172.21.15.96:0/4209897125 wait complete. 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.487+0000 7f202faac640 1 Processor -- start 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 -- start start 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 0x7f2028197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 0x7f2028198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f202810c380 con 0x7f20281022d0 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f202810c200 con 0x7f2028100720 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202faac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f202810c500 con 0x7f2028101910 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46864/0 (socket says 172.21.15.96:46864) 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 -- 172.21.15.96:0/2597358378 learned_addr learned my addr 172.21.15.96:0/2597358378 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202d848640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 0x7f2028197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202d047640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 0x7f2028198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 -- 172.21.15.96:0/2597358378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 msgr2=0x7f2028198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 0x7f2028198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 -- 172.21.15.96:0/2597358378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 msgr2=0x7f2028197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 0x7f2028197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 -- 172.21.15.96:0/2597358378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202819f6e0 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f202e049640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f202002c700 tx=0x7f2020032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.488+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020039040 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.489+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202819f970 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.489+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2020044070 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.489+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20200042a0 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.489+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202819feb0 con 0x7f20281022d0 2024-03-18T15:37:27.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.489+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20281a0080 con 0x7f20281022d0 2024-03-18T15:37:27.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.491+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2020003970 con 0x7f20281022d0 2024-03-18T15:37:27.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.491+0000 7f201effd640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 0x7f20140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:27.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.491+0000 7f202d848640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 0x7f20140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:27.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.491+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f20200f10d0 con 0x7f20281022d0 2024-03-18T15:37:27.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.492+0000 7f202d848640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 0x7f20140a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2028101770 tx=0x7f201800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:27.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.495+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f20200be7c0 con 0x7f20281022d0 2024-03-18T15:37:27.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.708+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20281a0250 con 0x7f20281022d0 2024-03-18T15:37:27.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.709+0000 7f201effd640 1 -- 172.21.15.96:0/2597358378 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f20200be0c0 con 0x7f20281022d0 2024-03-18T15:37:27.710 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:27.710 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:37:27.710 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 msgr2=0x7f20140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 0x7f20140a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2028101770 tx=0x7f201800b390 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 msgr2=0x7f202819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f202002c700 tx=0x7f2020032f20 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 shutdown_connections 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f20140a2800 0x7f20140a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20281022d0 0x7f202819f070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2028101910 0x7f2028198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 --2- 172.21.15.96:0/2597358378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2028100720 0x7f2028197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 >> 172.21.15.96:0/2597358378 conn(0x7f20280fbc90 msgr2=0x7f20280fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:27.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 shutdown_connections 2024-03-18T15:37:27.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:27.711+0000 7f202faac640 1 -- 172.21.15.96:0/2597358378 wait complete. 2024-03-18T15:37:28.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6832/3317488579,v1:172.21.15.193:6833/3317488579] up:boot 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: fsmap 1 up:standby 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: Deploying daemon mds.k on smithi137 2024-03-18T15:37:28.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:27 smithi137 ceph-mon[35868]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6832/3317488579,v1:172.21.15.193:6833/3317488579] up:boot 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: fsmap 1 up:standby 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:28.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:28.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:28.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: Deploying daemon mds.k on smithi137 2024-03-18T15:37:28.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:27 smithi193 ceph-mon[36702]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6832/3317488579,v1:172.21.15.193:6833/3317488579] up:boot 2024-03-18T15:37:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: fsmap 1 up:standby 2024-03-18T15:37:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:37:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: Deploying daemon mds.k on smithi137 2024-03-18T15:37:28.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:27 smithi096 ceph-mon[31930]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:28.846 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2597358378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:29.105 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 systemd[1]: Starting Ceph mds.k for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2597358378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2597358378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:29.536 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 podman[65347]: 2024-03-18 15:37:29.145611165 +0000 UTC m=+0.018256588 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:29.536 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 podman[65347]: 2024-03-18 15:37:29.283124019 +0000 UTC m=+0.155769428 container create 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-18T15:37:29.536 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 podman[65347]: 2024-03-18 15:37:29.536188648 +0000 UTC m=+0.408834072 container init 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T15:37:29.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 -- 172.21.15.96:0/4115107087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 msgr2=0x7fae38100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 --2- 172.21.15.96:0/4115107087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38100b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009900 tx=0x7fae2c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 -- 172.21.15.96:0/4115107087 shutdown_connections 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 --2- 172.21.15.96:0/4115107087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381022b0 0x7fae38108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 --2- 172.21.15.96:0/4115107087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381018f0 0x7fae38101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 --2- 172.21.15.96:0/4115107087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38100b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 -- 172.21.15.96:0/4115107087 >> 172.21.15.96:0/4115107087 conn(0x7fae380fbc70 msgr2=0x7fae380fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.834+0000 7fae3d836640 1 -- 172.21.15.96:0/4115107087 shutdown_connections 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 -- 172.21.15.96:0/4115107087 wait complete. 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 Processor -- start 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 -- start start 2024-03-18T15:37:29.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381018f0 0x7fae381980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 0x7fae3819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810c350 con 0x7fae38100700 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810c1d0 con 0x7fae381022b0 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.835+0000 7fae3d836640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae3810c4d0 con 0x7fae381018f0 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46882/0 (socket says 172.21.15.96:46882) 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 -- 172.21.15.96:0/108367137 learned_addr learned my addr 172.21.15.96:0/108367137 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae367fc640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381018f0 0x7fae381980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae377fe640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 0x7fae3819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 -- 172.21.15.96:0/108367137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381018f0 msgr2=0x7fae381980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381018f0 0x7fae381980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 -- 172.21.15.96:0/108367137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 msgr2=0x7fae3819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 0x7fae3819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:29.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 -- 172.21.15.96:0/108367137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3819f7d0 con 0x7fae38100700 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae377fe640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 0x7fae3819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae36ffd640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005ab0 tx=0x7fae2c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c041070 con 0x7fae38100700 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2c0054e0 con 0x7fae38100700 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3819fa60 con 0x7fae38100700 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c034440 con 0x7fae38100700 2024-03-18T15:37:29.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.836+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3819ff50 con 0x7fae38100700 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.837+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae381a0180 con 0x7fae38100700 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.838+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fae2c044070 con 0x7fae38100700 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.839+0000 7fae3c834640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 0x7fae200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.839+0000 7fae367fc640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 0x7fae200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.839+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fae2c0f0300 con 0x7fae38100700 2024-03-18T15:37:29.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.840+0000 7fae367fc640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 0x7fae200a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae28005fd0 tx=0x7fae28005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:29.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:29.844+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fae2c0bd840 con 0x7fae38100700 2024-03-18T15:37:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:29 smithi137 ceph-mon[35868]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:29 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:29.856 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 podman[65347]: 2024-03-18 15:37:29.539344442 +0000 UTC m=+0.411989862 container start 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:37:29.856 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 ceph-mds[65367]: -- 172.21.15.137:0/427761705 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d93a5b7a40 con 0x55d93a58b000 2024-03-18T15:37:29.856 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: starting mds.k at 2024-03-18T15:37:29.856 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 bash[65347]: 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed 2024-03-18T15:37:29.856 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:37:29 smithi137 systemd[1]: Started Ceph mds.k for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:30.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.064+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae381a0350 con 0x7fae38100700 2024-03-18T15:37:30.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.064+0000 7fae3c834640 1 -- 172.21.15.96:0/108367137 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fae2c0dd0c0 con 0x7fae38100700 2024-03-18T15:37:30.065 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:30.065 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:37:30.066 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-03-18T15:37:30.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 msgr2=0x7fae200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:30.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 0x7fae200a4cc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae28005fd0 tx=0x7fae28005850 comp rx=0 tx=0).stop 2024-03-18T15:37:30.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 msgr2=0x7fae38197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:30.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae2c005ab0 tx=0x7fae2c005dc0 comp rx=0 tx=0).stop 2024-03-18T15:37:30.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 shutdown_connections 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fae200a2800 0x7fae200a4cc0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae381022b0 0x7fae3819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae381018f0 0x7fae381980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 --2- 172.21.15.96:0/108367137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae38100700 0x7fae38197bb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 >> 172.21.15.96:0/108367137 conn(0x7fae380fbc70 msgr2=0x7fae380fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.066+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 shutdown_connections 2024-03-18T15:37:30.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:30.067+0000 7fae3d836640 1 -- 172.21.15.96:0/108367137 wait complete. 2024-03-18T15:37:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:29 smithi193 ceph-mon[36702]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:29 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:30.077 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:29 smithi096 ceph-mon[31930]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:30.077 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:30.077 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:29 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6832/753223976,v1:172.21.15.137:6833/753223976] up:boot 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: fsmap 2 up:standby 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: Deploying daemon mds.j on smithi096 2024-03-18T15:37:30.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/108367137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:31.080 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6832/753223976,v1:172.21.15.137:6833/753223976] up:boot 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: fsmap 2 up:standby 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: Deploying daemon mds.j on smithi096 2024-03-18T15:37:31.081 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/108367137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6832/753223976,v1:172.21.15.137:6833/753223976] up:boot 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: fsmap 2 up:standby 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: Deploying daemon mds.j on smithi096 2024-03-18T15:37:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/108367137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:31.957 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:31 smithi137 ceph-mon[35868]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:32.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:32.104 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:31 smithi096 ceph-mon[31930]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:32.105 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:31 smithi096 systemd[1]: Starting Ceph mds.j for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:31 smithi193 ceph-mon[36702]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 -- 172.21.15.96:0/4031781761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1022b0 msgr2=0x7ff1cc108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/4031781761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc108b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009ca0 tx=0x7ff1c402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 -- 172.21.15.96:0/4031781761 shutdown_connections 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/4031781761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc108b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/4031781761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/4031781761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 0x7ff1cc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 -- 172.21.15.96:0/4031781761 >> 172.21.15.96:0/4031781761 conn(0x7ff1cc0fbc90 msgr2=0x7ff1cc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 -- 172.21.15.96:0/4031781761 shutdown_connections 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.183+0000 7ff1d19cd640 1 -- 172.21.15.96:0/4031781761 wait complete. 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 Processor -- start 2024-03-18T15:37:32.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 -- start start 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 0x7ff1cc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d580 con 0x7ff1cc1018f0 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d400 con 0x7ff1cc100700 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1d19cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc10d700 con 0x7ff1cc1022b0 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46902/0 (socket says 172.21.15.96:46902) 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 -- 172.21.15.96:0/587653497 learned_addr learned my addr 172.21.15.96:0/587653497 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1caffd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 0x7ff1cc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1cb7fe640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 -- 172.21.15.96:0/587653497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1022b0 msgr2=0x7ff1cc19f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc19f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.184+0000 7ff1ca7fc640 1 -- 172.21.15.96:0/587653497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 msgr2=0x7ff1cc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1ca7fc640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 0x7ff1cc197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1ca7fc640 1 -- 172.21.15.96:0/587653497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19f7e0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1ca7fc640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b4a0 tx=0x7ff1c000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0015020 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c0011d10 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19fad0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c000f2e0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.185+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a0020 con 0x7ff1cc1018f0 2024-03-18T15:37:32.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.186+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc1079c0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.187+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7ff1c0018020 con 0x7ff1cc1018f0 2024-03-18T15:37:32.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.187+0000 7ff1d09cb640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 0x7ff1b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:32.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.188+0000 7ff1caffd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 0x7ff1b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:32.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.188+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff1c00ce270 con 0x7ff1cc1018f0 2024-03-18T15:37:32.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.188+0000 7ff1caffd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 0x7ff1b40a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc101750 tx=0x7ff1bc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:32.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.191+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff1c001f020 con 0x7ff1cc1018f0 2024-03-18T15:37:32.402 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 podman[72654]: 2024-03-18 15:37:32.022357497 +0000 UTC m=+0.017316909 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:32.402 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 podman[72654]: 2024-03-18 15:37:32.131460498 +0000 UTC m=+0.126419909 container create 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-03-18T15:37:32.402 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 podman[72654]: 2024-03-18 15:37:32.401894214 +0000 UTC m=+0.396853626 container init 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:37:32.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.412+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1cc02d1b0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.412+0000 7ff1d09cb640 1 -- 172.21.15.96:0/587653497 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7ff1c009b7b0 con 0x7ff1cc1018f0 2024-03-18T15:37:32.413 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:32.413 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:37:32.414 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-03-18T15:37:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.415+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 msgr2=0x7ff1b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.415+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 0x7ff1b40a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc101750 tx=0x7ff1bc009110 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.415+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 msgr2=0x7ff1cc198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.415+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b4a0 tx=0x7ff1c000b8a0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.415+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 shutdown_connections 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1b40a2800 0x7ff1b40a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1cc1022b0 0x7ff1cc19f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1cc1018f0 0x7ff1cc198100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 --2- 172.21.15.96:0/587653497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1cc100700 0x7ff1cc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 >> 172.21.15.96:0/587653497 conn(0x7ff1cc0fbc90 msgr2=0x7ff1cc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 shutdown_connections 2024-03-18T15:37:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:32.416+0000 7ff1d19cd640 1 -- 172.21.15.96:0/587653497 wait complete. 2024-03-18T15:37:32.686 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 podman[72654]: 2024-03-18 15:37:32.404867089 +0000 UTC m=+0.399826509 container start 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:37:32.686 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mds[72712]: -- 172.21.15.96:0/3000707089 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f18a41a40 con 0x564f18a14c00 2024-03-18T15:37:32.686 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j[72708]: starting mds.j at 2024-03-18T15:37:32.686 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 bash[72654]: 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 2024-03-18T15:37:32.686 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:37:32 smithi096 systemd[1]: Started Ceph mds.j for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/587653497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:32.985 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/587653497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/587653497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:33 smithi096 ceph-mon[31930]: Deploying daemon mds.d on smithi096 2024-03-18T15:37:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:33 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6834/1518442933,v1:172.21.15.96:6835/1518442933] up:boot 2024-03-18T15:37:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:33 smithi096 ceph-mon[31930]: fsmap 3 up:standby 2024-03-18T15:37:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T15:37:33.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:33 smithi096 ceph-mon[31930]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:33 smithi137 ceph-mon[35868]: Deploying daemon mds.d on smithi096 2024-03-18T15:37:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:33 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6834/1518442933,v1:172.21.15.96:6835/1518442933] up:boot 2024-03-18T15:37:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:33 smithi137 ceph-mon[35868]: fsmap 3 up:standby 2024-03-18T15:37:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:33 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T15:37:34.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:33 smithi137 ceph-mon[35868]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:33 smithi193 ceph-mon[36702]: Deploying daemon mds.d on smithi096 2024-03-18T15:37:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:33 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6834/1518442933,v1:172.21.15.96:6835/1518442933] up:boot 2024-03-18T15:37:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:33 smithi193 ceph-mon[36702]: fsmap 3 up:standby 2024-03-18T15:37:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:33 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-18T15:37:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:33 smithi193 ceph-mon[36702]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:34.432 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 -- 172.21.15.96:0/597660282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 msgr2=0x7f82f8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 --2- 172.21.15.96:0/597660282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8102ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009900 tx=0x7f82e802bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 -- 172.21.15.96:0/597660282 shutdown_connections 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 --2- 172.21.15.96:0/597660282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f81033e0 0x7f82f8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 --2- 172.21.15.96:0/597660282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8102ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 --2- 172.21.15.96:0/597660282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f8101830 0x7f82f8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 -- 172.21.15.96:0/597660282 >> 172.21.15.96:0/597660282 conn(0x7f82f80fcd80 msgr2=0x7f82f80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 -- 172.21.15.96:0/597660282 shutdown_connections 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 -- 172.21.15.96:0/597660282 wait complete. 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.581+0000 7f82fdadc640 1 Processor -- start 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- start start 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 0x7f82f8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 0x7f82f819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810d4b0 con 0x7f82f8102a20 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810d330 con 0x7f82f8101830 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810d630 con 0x7f82f81033e0 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51950/0 (socket says 172.21.15.96:51950) 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 -- 172.21.15.96:0/3076428761 learned_addr learned my addr 172.21.15.96:0/3076428761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:34.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f7fff640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 0x7f82f819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f77fe640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 0x7f82f8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 -- 172.21.15.96:0/3076428761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 msgr2=0x7f82f819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 0x7f82f819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 -- 172.21.15.96:0/3076428761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 msgr2=0x7f82f8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 0x7f82f8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 -- 172.21.15.96:0/3076428761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f819f770 con 0x7f82f8102a20 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f7fff640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 0x7f82f819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f77fe640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 0x7f82f8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f6ffd640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009230 tx=0x7f82e8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8041070 con 0x7f82f8102a20 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f819fa00 con 0x7f82f8102a20 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e80054e0 con 0x7f82f8102a20 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8034440 con 0x7f82f8102a20 2024-03-18T15:37:34.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.582+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f819ff40 con 0x7f82f8102a20 2024-03-18T15:37:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.584+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8108b50 con 0x7f82f8102a20 2024-03-18T15:37:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.584+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f82e8044070 con 0x7f82f8102a20 2024-03-18T15:37:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.585+0000 7f82f4ff9640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 0x7f82e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:34.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.585+0000 7f82f77fe640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 0x7f82e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:34.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.585+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82e80f0370 con 0x7f82f8102a20 2024-03-18T15:37:34.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.585+0000 7f82f77fe640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 0x7f82e00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0058c0 tx=0x7f82ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:34.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.590+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f82e80bd8b0 con 0x7f82f8102a20 2024-03-18T15:37:34.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.841+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f802d360 con 0x7f82f8102a20 2024-03-18T15:37:34.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.842+0000 7f82f4ff9640 1 -- 172.21.15.96:0/3076428761 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f82e80bd1b0 con 0x7f82f8102a20 2024-03-18T15:37:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:34.843 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:37:34.843 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 4 2024-03-18T15:37:34.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 msgr2=0x7f82e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:34.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 0x7f82e00a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0058c0 tx=0x7f82ec005850 comp rx=0 tx=0).stop 2024-03-18T15:37:34.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 msgr2=0x7f82f8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:34.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009230 tx=0x7f82e8005dd0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 shutdown_connections 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82e00a2800 0x7f82e00a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82f81033e0 0x7f82f819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f8102a20 0x7f82f8198090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 --2- 172.21.15.96:0/3076428761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82f8101830 0x7f82f8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 >> 172.21.15.96:0/3076428761 conn(0x7f82f80fcd80 msgr2=0x7f82f8105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 shutdown_connections 2024-03-18T15:37:34.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:34.844+0000 7f82fdadc640 1 -- 172.21.15.96:0/3076428761 wait complete. 2024-03-18T15:37:34.857 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:34 smithi096 systemd[1]: Starting Ceph mds.d for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:35.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:34 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3076428761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:35.121 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:34 smithi096 podman[73048]: 2024-03-18 15:37:34.852485632 +0000 UTC m=+0.019081532 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:35.121 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:34 smithi096 podman[73048]: 2024-03-18 15:37:34.980804607 +0000 UTC m=+0.147400499 container create 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-03-18T15:37:35.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:34 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3076428761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:35.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:34 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3076428761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 podman[73048]: 2024-03-18 15:37:35.233966266 +0000 UTC m=+0.400562172 container init 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 podman[73048]: 2024-03-18 15:37:35.236621389 +0000 UTC m=+0.403217292 container start 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 ceph-mds[73069]: -- 172.21.15.96:0/1030660358 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d22f601a40 con 0x55d22f5d4c00 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d[73065]: starting mds.d at 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 bash[73048]: 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d 2024-03-18T15:37:35.686 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:37:35 smithi096 systemd[1]: Started Ceph mds.d for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:36.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: Deploying daemon mds.e on smithi137 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6836/2891815137,v1:172.21.15.96:6837/2891815137] up:boot 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: fsmap 4 up:standby 2024-03-18T15:37:36.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: Deploying daemon mds.e on smithi137 2024-03-18T15:37:36.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6836/2891815137,v1:172.21.15.96:6837/2891815137] up:boot 2024-03-18T15:37:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: fsmap 4 up:standby 2024-03-18T15:37:36.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T15:37:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: Deploying daemon mds.e on smithi137 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6836/2891815137,v1:172.21.15.96:6837/2891815137] up:boot 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: fsmap 4 up:standby 2024-03-18T15:37:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-18T15:37:36.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/4166775787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 msgr2=0x7fd6b8109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/4166775787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b8109c40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009ca0 tx=0x7fd6b002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/4166775787 shutdown_connections 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/4166775787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b8109c40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/4166775787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 0x7fd6b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/4166775787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 0x7fd6b8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/4166775787 >> 172.21.15.96:0/4166775787 conn(0x7fd6b80fcd90 msgr2=0x7fd6b80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/4166775787 shutdown_connections 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.973+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/4166775787 wait complete. 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 Processor -- start 2024-03-18T15:37:36.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 -- start start 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 0x7fd6b8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 0x7fd6b8198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810d520 con 0x7fd6b81033b0 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810d3a0 con 0x7fd6b8101800 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6bdc0b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b810d6a0 con 0x7fd6b81029f0 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52000/0 (socket says 172.21.15.96:52000) 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 -- 172.21.15.96:0/3686195011 learned_addr learned my addr 172.21.15.96:0/3686195011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b77fe640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 0x7fd6b8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b6ffd640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 0x7fd6b8198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 -- 172.21.15.96:0/3686195011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 msgr2=0x7fd6b8198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 0x7fd6b8198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 -- 172.21.15.96:0/3686195011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 msgr2=0x7fd6b8197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 0x7fd6b8197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:36.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.974+0000 7fd6b7fff640 1 -- 172.21.15.96:0/3686195011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b819f750 con 0x7fd6b81033b0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6b7fff640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005e30 tx=0x7fd6b0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0041020 con 0x7fd6b81033b0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b819f9e0 con 0x7fd6b81033b0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0034cd0 con 0x7fd6b81033b0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0039040 con 0x7fd6b81033b0 2024-03-18T15:37:36.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.975+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b819fed0 con 0x7fd6b81033b0 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.976+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b8108ac0 con 0x7fd6b81033b0 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.977+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd6b0044020 con 0x7fd6b81033b0 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.978+0000 7fd6b4ff9640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 0x7fd6a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.978+0000 7fd6b77fe640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 0x7fd6a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.978+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd6b00f0a00 con 0x7fd6b81033b0 2024-03-18T15:37:36.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.978+0000 7fd6b77fe640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 0x7fd6a00a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8102850 tx=0x7fd6a8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:36.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:36.981+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd6b004b020 con 0x7fd6b81033b0 2024-03-18T15:37:37.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.194+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6b802d1b0 con 0x7fd6b81033b0 2024-03-18T15:37:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.195+0000 7fd6b4ff9640 1 -- 172.21.15.96:0/3686195011 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fd6b00bdf40 con 0x7fd6b81033b0 2024-03-18T15:37:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:37.196 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:37:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 5 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 msgr2=0x7fd6a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 0x7fd6a00a4cc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8102850 tx=0x7fd6a8009110 comp rx=0 tx=0).stop 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 msgr2=0x7fd6b819f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0005e30 tx=0x7fd6b0032ec0 comp rx=0 tx=0).stop 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 shutdown_connections 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6a00a2800 0x7fd6a00a4cc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:37.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6b81033b0 0x7fd6b819f0e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:37.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b81029f0 0x7fd6b8198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:37.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 --2- 172.21.15.96:0/3686195011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6b8101800 0x7fd6b8197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:37.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 >> 172.21.15.96:0/3686195011 conn(0x7fd6b80fcd90 msgr2=0x7fd6b8105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:37.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 shutdown_connections 2024-03-18T15:37:37.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:37.197+0000 7fd6bdc0b640 1 -- 172.21.15.96:0/3686195011 wait complete. 2024-03-18T15:37:37.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:37 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3686195011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:37.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:37 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3686195011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:37.856 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:37 smithi137 systemd[1]: Starting Ceph mds.e for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:37.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:37 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3686195011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:38.157 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:37 smithi137 podman[65669]: 2024-03-18 15:37:37.76802974 +0000 UTC m=+0.018026915 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:38.157 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:37 smithi137 podman[65669]: 2024-03-18 15:37:37.904781676 +0000 UTC m=+0.154778852 container create dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:37:38.157 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 podman[65669]: 2024-03-18 15:37:38.157469698 +0000 UTC m=+0.407466868 container init dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T15:37:38.433 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 podman[65669]: 2024-03-18 15:37:38.16002272 +0000 UTC m=+0.410019885 container start dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T15:37:38.433 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mds[65687]: -- 172.21.15.137:0/74645921 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd54169a40 con 0x55dd5413c800 2024-03-18T15:37:38.433 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: starting mds.e at 2024-03-18T15:37:38.433 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 bash[65669]: dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b 2024-03-18T15:37:38.433 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:37:38 smithi137 systemd[1]: Started Ceph mds.e for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:38.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:39.209 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:39.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 -- 172.21.15.96:0/1017584289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0100710 msgr2=0x7f47e0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 --2- 172.21.15.96:0/1017584289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0100710 0x7f47e0100b30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009900 tx=0x7f47d402bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 -- 172.21.15.96:0/1017584289 shutdown_connections 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 --2- 172.21.15.96:0/1017584289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e0108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 --2- 172.21.15.96:0/1017584289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0101900 0x7f47e0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 --2- 172.21.15.96:0/1017584289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0100710 0x7f47e0100b30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 -- 172.21.15.96:0/1017584289 >> 172.21.15.96:0/1017584289 conn(0x7f47e00fbcc0 msgr2=0x7f47e00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 -- 172.21.15.96:0/1017584289 shutdown_connections 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.321+0000 7f47e6eea640 1 -- 172.21.15.96:0/1017584289 wait complete. 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 Processor -- start 2024-03-18T15:37:39.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 -- start start 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 0x7f47e006a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47e010c360 con 0x7f47e0101900 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47e010c1e0 con 0x7f47e0100710 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e6eea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47e010c4e0 con 0x7f47e01022c0 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52028/0 (socket says 172.21.15.96:52028) 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 -- 172.21.15.96:0/36113961 learned_addr learned my addr 172.21.15.96:0/36113961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e4c86640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 0x7f47e006a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e5487640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:39.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 -- 172.21.15.96:0/36113961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 msgr2=0x7f47e019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 -- 172.21.15.96:0/36113961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 msgr2=0x7f47e006a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 0x7f47e006a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 -- 172.21.15.96:0/36113961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e019f7c0 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e5487640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47e4c86640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 0x7f47e006a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.322+0000 7f47dffff640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d950 tx=0x7f47cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.323+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc017070 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.323+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc009c30 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.323+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e019fa50 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.323+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00f6b0 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.323+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e019ffa0 con 0x7f47e0101900 2024-03-18T15:37:39.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.324+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e0107ab0 con 0x7f47e0101900 2024-03-18T15:37:39.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.325+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f47cc0109d0 con 0x7f47e0101900 2024-03-18T15:37:39.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.325+0000 7f47ddffb640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 0x7f47d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:39.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.326+0000 7f47e4c86640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 0x7f47d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:39.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.326+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47cc0ce320 con 0x7f47e0101900 2024-03-18T15:37:39.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.326+0000 7f47e4c86640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 0x7f47d00a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c7b0 tx=0x7f47d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:39.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.330+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f47cc009de0 con 0x7f47e0101900 2024-03-18T15:37:39.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.543+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47e002d380 con 0x7f47e0101900 2024-03-18T15:37:39.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.544+0000 7f47ddffb640 1 -- 172.21.15.96:0/36113961 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f47cc09b860 con 0x7f47e0101900 2024-03-18T15:37:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:39.545 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-03-18T15:37:39.545 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 msgr2=0x7f47d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 0x7f47d00a4cc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c7b0 tx=0x7f47d4005fd0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 msgr2=0x7f47e019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d950 tx=0x7f47cc00de10 comp rx=0 tx=0).stop 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 shutdown_connections 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47d00a2800 0x7f47d00a4cc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47e01022c0 0x7f47e019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47e0101900 0x7f47e019c890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 --2- 172.21.15.96:0/36113961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47e0100710 0x7f47e006a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:39.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 >> 172.21.15.96:0/36113961 conn(0x7f47e00fbcc0 msgr2=0x7f47e0104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:39.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 shutdown_connections 2024-03-18T15:37:39.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:39.546+0000 7f47e6eea640 1 -- 172.21.15.96:0/36113961 wait complete. 2024-03-18T15:37:39.557 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:39 smithi096 ceph-mon[31930]: Deploying daemon mds.f on smithi193 2024-03-18T15:37:39.557 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:39 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6834/3504042144,v1:172.21.15.137:6835/3504042144] up:boot 2024-03-18T15:37:39.557 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:39 smithi096 ceph-mon[31930]: fsmap 5 up:standby 2024-03-18T15:37:39.557 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:39 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:37:39.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:39 smithi193 ceph-mon[36702]: Deploying daemon mds.f on smithi193 2024-03-18T15:37:39.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:39 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6834/3504042144,v1:172.21.15.137:6835/3504042144] up:boot 2024-03-18T15:37:39.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:39 smithi193 ceph-mon[36702]: fsmap 5 up:standby 2024-03-18T15:37:39.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:39 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:37:39.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:39 smithi137 ceph-mon[35868]: Deploying daemon mds.f on smithi193 2024-03-18T15:37:39.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:39 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6834/3504042144,v1:172.21.15.137:6835/3504042144] up:boot 2024-03-18T15:37:39.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:39 smithi137 ceph-mon[35868]: fsmap 5 up:standby 2024-03-18T15:37:39.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:39 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:37:40.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:40 smithi193 ceph-mon[36702]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:40.589 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:40 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/36113961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:40 smithi137 ceph-mon[35868]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:40 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/36113961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:40 smithi096 ceph-mon[31930]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:40 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/36113961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:41.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:41.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 -- 172.21.15.96:0/2900544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc1018f0 msgr2=0x7fb6cc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:41.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/2900544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc101d70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009900 tx=0x7fb6c002bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:41.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 -- 172.21.15.96:0/2900544640 shutdown_connections 2024-03-18T15:37:41.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/2900544640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1022b0 0x7fb6cc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/2900544640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc101d70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/2900544640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc100700 0x7fb6cc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 -- 172.21.15.96:0/2900544640 >> 172.21.15.96:0/2900544640 conn(0x7fb6cc0fbc70 msgr2=0x7fb6cc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:41.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 -- 172.21.15.96:0/2900544640 shutdown_connections 2024-03-18T15:37:41.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 -- 172.21.15.96:0/2900544640 wait complete. 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.673+0000 7fb6d08c0640 1 Processor -- start 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 -- start start 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc1022b0 0x7fb6cc19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:41.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc10c380 con 0x7fb6cc100700 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc10c200 con 0x7fb6cc1018f0 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6d08c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc10c500 con 0x7fb6cc1022b0 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52048/0 (socket says 172.21.15.96:52048) 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 -- 172.21.15.96:0/4110706403 learned_addr learned my addr 172.21.15.96:0/4110706403 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6cad9d640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc1022b0 0x7fb6cc19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6c9d9b640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 -- 172.21.15.96:0/4110706403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc1022b0 msgr2=0x7fb6cc19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc1022b0 0x7fb6cc19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 -- 172.21.15.96:0/4110706403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 msgr2=0x7fb6cc196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 -- 172.21.15.96:0/4110706403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc19e670 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6c9d9b640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6ca59c640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400eaa0 tx=0x7fb6c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.674+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4016070 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.675+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6c4003e50 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.675+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc19e960 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.675+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40116d0 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.675+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc19ee60 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.676+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc107aa0 con 0x7fb6cc100700 2024-03-18T15:37:41.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.677+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb6c4019070 con 0x7fb6cc100700 2024-03-18T15:37:41.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.677+0000 7fb6bb7fe640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 0x7fb6b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:41.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.677+0000 7fb6c9d9b640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 0x7fb6b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:41.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.678+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb6c40ce6a0 con 0x7fb6cc100700 2024-03-18T15:37:41.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.678+0000 7fb6c9d9b640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 0x7fb6b40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0036040 tx=0x7fb6c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:41.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.682+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb6c409bbe0 con 0x7fb6cc100700 2024-03-18T15:37:41.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.894+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6cc02d1b0 con 0x7fb6cc100700 2024-03-18T15:37:41.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.894+0000 7fb6bb7fe640 1 -- 172.21.15.96:0/4110706403 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7fb6c409b4e0 con 0x7fb6cc100700 2024-03-18T15:37:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:41.898 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 msgr2=0x7fb6b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 0x7fb6b40a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0036040 tx=0x7fb6c0000bc0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 msgr2=0x7fb6cc196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400eaa0 tx=0x7fb6c400ef60 comp rx=0 tx=0).stop 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 shutdown_connections 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6b40a2800 0x7fb6b40a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6cc1022b0 0x7fb6cc19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb6cc1018f0 0x7fb6cc196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 --2- 172.21.15.96:0/4110706403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6cc100700 0x7fb6cc196a50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:41.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 >> 172.21.15.96:0/4110706403 conn(0x7fb6cc0fbc70 msgr2=0x7fb6cc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:41.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 shutdown_connections 2024-03-18T15:37:41.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:41.896+0000 7fb6d08c0640 1 -- 172.21.15.96:0/4110706403 wait complete. 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:42.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: Deploying daemon mds.c on smithi193 2024-03-18T15:37:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:boot 2024-03-18T15:37:42.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: fsmap 6 up:standby 2024-03-18T15:37:42.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:37:42.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:42 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4110706403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:42.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: Deploying daemon mds.c on smithi193 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:boot 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: fsmap 6 up:standby 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:37:42.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:42 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4110706403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:42.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: Deploying daemon mds.c on smithi193 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:boot 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: fsmap 6 up:standby 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:37:42.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:42 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4110706403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:43.347 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 systemd[1]: Starting Ceph mds.c for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:43.617 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 podman[67675]: 2024-03-18 15:37:43.249692166 +0000 UTC m=+0.017801261 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:43.617 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 podman[67675]: 2024-03-18 15:37:43.381231856 +0000 UTC m=+0.149340940 container create f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-18T15:37:43.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 podman[67675]: 2024-03-18 15:37:43.617780267 +0000 UTC m=+0.385889372 container init f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 podman[67675]: 2024-03-18 15:37:43.620440735 +0000 UTC m=+0.388549812 container start f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 ceph-mds[67695]: -- 172.21.15.193:0/1009969277 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b0ba8fa40 con 0x559b0ba62800 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: starting mds.c at 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 bash[67675]: f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 2024-03-18T15:37:43.918 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:37:43 smithi193 systemd[1]: Started Ceph mds.c for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:44.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/3187063654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f141009d0 msgr2=0x7f2f14100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/3187063654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f141009d0 0x7f2f14100e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2f04009900 tx=0x7f2f0402bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/3187063654 shutdown_connections 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/3187063654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 0x7f2f14108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/3187063654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f141009d0 0x7f2f14100e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/3187063654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f14101ca0 0x7f2f141020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/3187063654 >> 172.21.15.96:0/3187063654 conn(0x7f2f140fb400 msgr2=0x7f2f140fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/3187063654 shutdown_connections 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.020+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/3187063654 wait complete. 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 Processor -- start 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 -- start start 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f141009d0 0x7f2f14100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 0x7f2f1419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:44.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410c380 con 0x7f2f14101ca0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410c200 con 0x7f2f14101390 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1bcb2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1410c500 con 0x7f2f141009d0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42450/0 (socket says 172.21.15.96:42450) 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 -- 172.21.15.96:0/151700751 learned_addr learned my addr 172.21.15.96:0/151700751 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1924d640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 0x7f2f1419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f19a4e640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f141009d0 0x7f2f14100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 -- 172.21.15.96:0/151700751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f141009d0 msgr2=0x7f2f14100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f141009d0 0x7f2f14100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 -- 172.21.15.96:0/151700751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 msgr2=0x7f2f1419c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 0x7f2f1419c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.021+0000 7f2f1a24f640 1 -- 172.21.15.96:0/151700751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1419f7a0 con 0x7f2f14101ca0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f1a24f640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00eb10 tx=0x7f2f0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:44.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004b20 con 0x7f2f14101ca0 2024-03-18T15:37:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f0c004cc0 con 0x7f2f14101ca0 2024-03-18T15:37:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1419fa30 con 0x7f2f14101ca0 2024-03-18T15:37:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c010760 con 0x7f2f14101ca0 2024-03-18T15:37:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.022+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1419ff00 con 0x7f2f14101ca0 2024-03-18T15:37:44.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.023+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14107b90 con 0x7f2f14101ca0 2024-03-18T15:37:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.024+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f2f0c005020 con 0x7f2f14101ca0 2024-03-18T15:37:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.024+0000 7f2f0affd640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 0x7f2f000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.025+0000 7f2f19a4e640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 0x7f2f000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.025+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2f0c0cebe0 con 0x7f2f14101ca0 2024-03-18T15:37:44.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.025+0000 7f2f19a4e640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 0x7f2f000a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005fd0 tx=0x7f2f10005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:44.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.029+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2f0c09c120 con 0x7f2f14101ca0 2024-03-18T15:37:44.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.241+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f1402d200 con 0x7f2f14101ca0 2024-03-18T15:37:44.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.242+0000 7f2f0affd640 1 -- 172.21.15.96:0/151700751 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f2f0c09ba20 con 0x7f2f14101ca0 2024-03-18T15:37:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:44.243 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-03-18T15:37:44.243 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 7 2024-03-18T15:37:44.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 msgr2=0x7f2f000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:44.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 0x7f2f000a4cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f10005fd0 tx=0x7f2f10005850 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 msgr2=0x7f2f1419ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00eb10 tx=0x7f2f0c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 shutdown_connections 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2f000a2800 0x7f2f000a4cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f14101ca0 0x7f2f1419ee90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f14101390 0x7f2f1419c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 --2- 172.21.15.96:0/151700751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f141009d0 0x7f2f14100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.243+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 >> 172.21.15.96:0/151700751 conn(0x7f2f140fb400 msgr2=0x7f2f14104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.244+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 shutdown_connections 2024-03-18T15:37:44.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:44.244+0000 7f2f1bcb2640 1 -- 172.21.15.96:0/151700751 wait complete. 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:44.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:44 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/151700751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:44.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:44 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/151700751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:44 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/151700751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:45.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:45 smithi137 ceph-mon[35868]: Deploying daemon mds.h on smithi137 2024-03-18T15:37:45.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:45 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:boot 2024-03-18T15:37:45.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:45 smithi137 ceph-mon[35868]: fsmap 7 up:standby 2024-03-18T15:37:45.839 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:45 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:37:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:45 smithi193 ceph-mon[36702]: Deploying daemon mds.h on smithi137 2024-03-18T15:37:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:45 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:boot 2024-03-18T15:37:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:45 smithi193 ceph-mon[36702]: fsmap 7 up:standby 2024-03-18T15:37:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:45 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:37:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:45 smithi096 ceph-mon[31930]: Deploying daemon mds.h on smithi137 2024-03-18T15:37:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:45 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:boot 2024-03-18T15:37:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:45 smithi096 ceph-mon[31930]: fsmap 7 up:standby 2024-03-18T15:37:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:45 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:37:46.105 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 systemd[1]: Starting Ceph mds.h for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:46.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 -- 172.21.15.96:0/588669655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30681018f0 msgr2=0x7f3068101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 --2- 172.21.15.96:0/588669655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30681018f0 0x7f3068101d70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3050009900 tx=0x7f305002bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 -- 172.21.15.96:0/588669655 shutdown_connections 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 --2- 172.21.15.96:0/588669655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 0x7f3068108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 --2- 172.21.15.96:0/588669655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30681018f0 0x7f3068101d70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 --2- 172.21.15.96:0/588669655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3068100700 0x7f3068100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 -- 172.21.15.96:0/588669655 >> 172.21.15.96:0/588669655 conn(0x7f30680fbc90 msgr2=0x7f30680fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 -- 172.21.15.96:0/588669655 shutdown_connections 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.369+0000 7f306d8b2640 1 -- 172.21.15.96:0/588669655 wait complete. 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 Processor -- start 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 -- start start 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30681018f0 0x7f3068198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 0x7f306819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d4a0 con 0x7f3068100700 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d320 con 0x7f30681018f0 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f306d8b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f306810d620 con 0x7f30681022b0 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42468/0 (socket says 172.21.15.96:42468) 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 -- 172.21.15.96:0/141515071 learned_addr learned my addr 172.21.15.96:0/141515071 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f30677fe640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 0x7f306819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f30667fc640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30681018f0 0x7f3068198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 -- 172.21.15.96:0/141515071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 msgr2=0x7f306819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 0x7f306819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 -- 172.21.15.96:0/141515071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30681018f0 msgr2=0x7f3068198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30681018f0 0x7f3068198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.370+0000 7f3066ffd640 1 -- 172.21.15.96:0/141515071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306819f720 con 0x7f3068100700 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f3066ffd640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f306000ebc0 tx=0x7f306000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060003b70 con 0x7f3068100700 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060003d10 con 0x7f3068100700 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306819fa10 con 0x7f3068100700 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060019980 con 0x7f3068100700 2024-03-18T15:37:46.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.371+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306819ff30 con 0x7f3068100700 2024-03-18T15:37:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.372+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3068107a20 con 0x7f3068100700 2024-03-18T15:37:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.373+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f3060019b20 con 0x7f3068100700 2024-03-18T15:37:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.373+0000 7f306c8b0640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 0x7f30540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.373+0000 7f30667fc640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 0x7f30540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:46.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.373+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f30600cee50 con 0x7f3068100700 2024-03-18T15:37:46.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.374+0000 7f30667fc640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 0x7f30540a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f3050000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:46.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.378+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f306009c390 con 0x7f3068100700 2024-03-18T15:37:46.504 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 podman[65979]: 2024-03-18 15:37:46.145111074 +0000 UTC m=+0.017281404 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:46.505 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 podman[65979]: 2024-03-18 15:37:46.260198313 +0000 UTC m=+0.132368649 container create 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T15:37:46.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.596+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f306802d230 con 0x7f3068100700 2024-03-18T15:37:46.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.597+0000 7f306c8b0640 1 -- 172.21.15.96:0/141515071 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f306009bc90 con 0x7f3068100700 2024-03-18T15:37:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:46.601 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 msgr2=0x7f30540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 0x7f30540a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f3050000bc0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 msgr2=0x7f3068197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f306000ebc0 tx=0x7f306000c6a0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 shutdown_connections 2024-03-18T15:37:46.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30540a2800 0x7f30540a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30681022b0 0x7f306819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30681018f0 0x7f3068198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 --2- 172.21.15.96:0/141515071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3068100700 0x7f3068197b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 >> 172.21.15.96:0/141515071 conn(0x7f30680fbc90 msgr2=0x7f3068104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 shutdown_connections 2024-03-18T15:37:46.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:46.599+0000 7f306d8b2640 1 -- 172.21.15.96:0/141515071 wait complete. 2024-03-18T15:37:46.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:46 smithi096 ceph-mon[31930]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:46 smithi137 ceph-mon[35868]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 podman[65979]: 2024-03-18 15:37:46.529579366 +0000 UTC m=+0.401749688 container init 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 podman[65979]: 2024-03-18 15:37:46.532561236 +0000 UTC m=+0.404731550 container start 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 ceph-mds[65998]: -- 172.21.15.137:0/3116387132 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56475cda1a40 con 0x56475cd74800 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: starting mds.h at 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 bash[65979]: 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 2024-03-18T15:37:46.856 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:37:46 smithi137 systemd[1]: Started Ceph mds.h for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:46.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:46 smithi193 ceph-mon[36702]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:47.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/141515071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:47.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:47 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/141515071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:47 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/141515071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:47.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:47.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:47 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:48.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:48 smithi193 ceph-mon[36702]: Deploying daemon mds.i on smithi193 2024-03-18T15:37:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:48 smithi193 ceph-mon[36702]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:48 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:boot 2024-03-18T15:37:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:48 smithi193 ceph-mon[36702]: fsmap 8 up:standby 2024-03-18T15:37:48.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:48 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:37:48.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- 172.21.15.96:0/3437916048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1018f0 msgr2=0x7f7fac101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/3437916048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1018f0 0x7f7fac101d70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009900 tx=0x7f7f9c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- 172.21.15.96:0/3437916048 shutdown_connections 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/3437916048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/3437916048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac1018f0 0x7f7fac101d70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/3437916048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac100700 0x7f7fac100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- 172.21.15.96:0/3437916048 >> 172.21.15.96:0/3437916048 conn(0x7f7fac0fbc70 msgr2=0x7f7fac0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- 172.21.15.96:0/3437916048 shutdown_connections 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- 172.21.15.96:0/3437916048 wait complete. 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 Processor -- start 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- start start 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac1018f0 0x7f7fac197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac10d4a0 con 0x7f7fac100700 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac10d320 con 0x7f7fac1022b0 2024-03-18T15:37:48.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.724+0000 7f7fb42a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac10d620 con 0x7f7fac1018f0 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42476/0 (socket says 172.21.15.96:42476) 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 -- 172.21.15.96:0/83387141 learned_addr learned my addr 172.21.15.96:0/83387141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb183c640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac1018f0 0x7f7fac197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb283e640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 -- 172.21.15.96:0/83387141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac1018f0 msgr2=0x7f7fac197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac1018f0 0x7f7fac197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 -- 172.21.15.96:0/83387141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 msgr2=0x7f7fac19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 -- 172.21.15.96:0/83387141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac19f690 con 0x7f7fac100700 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb283e640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb203d640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c970 tx=0x7f7fa800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8004220 con 0x7f7fac100700 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac19f980 con 0x7f7fac100700 2024-03-18T15:37:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa80043c0 con 0x7f7fac100700 2024-03-18T15:37:48.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800f450 con 0x7f7fac100700 2024-03-18T15:37:48.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.725+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac19fea0 con 0x7f7fac100700 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.726+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fac0685d0 con 0x7f7fac100700 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.727+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f7fa80079b0 con 0x7f7fac100700 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.728+0000 7f7fa37fe640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 0x7f7f980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.728+0000 7f7fb183c640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 0x7f7f980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.728+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7fa80ce340 con 0x7f7fac100700 2024-03-18T15:37:48.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.729+0000 7f7fb183c640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 0x7f7f980a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005ab0 tx=0x7f7f9c004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:48.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.733+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7fa8014030 con 0x7f7fac100700 2024-03-18T15:37:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:48 smithi137 ceph-mon[35868]: Deploying daemon mds.i on smithi193 2024-03-18T15:37:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:48 smithi137 ceph-mon[35868]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:48 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:boot 2024-03-18T15:37:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:48 smithi137 ceph-mon[35868]: fsmap 8 up:standby 2024-03-18T15:37:48.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:48 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:37:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:48 smithi096 ceph-mon[31930]: Deploying daemon mds.i on smithi193 2024-03-18T15:37:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:48 smithi096 ceph-mon[31930]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:48 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:boot 2024-03-18T15:37:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:48 smithi096 ceph-mon[31930]: fsmap 8 up:standby 2024-03-18T15:37:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:48 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:37:48.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.947+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fac107a40 con 0x7f7fac100700 2024-03-18T15:37:48.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.948+0000 7f7fa37fe640 1 -- 172.21.15.96:0/83387141 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f7fa809b880 con 0x7f7fac100700 2024-03-18T15:37:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:48.949 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-03-18T15:37:48.949 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 msgr2=0x7f7f980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 0x7f7f980a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005ab0 tx=0x7f7f9c004f10 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 msgr2=0x7f7fac197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800c970 tx=0x7f7fa800ce30 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 shutdown_connections 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7f980a2800 0x7f7f980a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7fac1022b0 0x7f7fac19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fac1018f0 0x7f7fac197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 --2- 172.21.15.96:0/83387141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fac100700 0x7f7fac197a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 >> 172.21.15.96:0/83387141 conn(0x7f7fac0fbc70 msgr2=0x7f7fac104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:48.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 shutdown_connections 2024-03-18T15:37:48.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:48.950+0000 7f7fb42a1640 1 -- 172.21.15.96:0/83387141 wait complete. 2024-03-18T15:37:49.130 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:48 smithi193 systemd[1]: Starting Ceph mds.i for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:49.405 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 podman[67980]: 2024-03-18 15:37:49.032405584 +0000 UTC m=+0.017607861 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:49.405 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 podman[67980]: 2024-03-18 15:37:49.144517628 +0000 UTC m=+0.129719930 container create d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:49 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/83387141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 podman[67980]: 2024-03-18 15:37:49.405298694 +0000 UTC m=+0.390500964 container init d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 podman[67980]: 2024-03-18 15:37:49.407923397 +0000 UTC m=+0.393125662 container start d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 ceph-mds[68000]: -- 172.21.15.193:0/2076036127 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e46ff7a40 con 0x563e46fca800 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: starting mds.i at 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 bash[67980]: d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 2024-03-18T15:37:49.668 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:37:49 smithi193 systemd[1]: Started Ceph mds.i for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:49.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:49 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/83387141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:49.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:49 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/83387141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:boot 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: fsmap 9 up:standby 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:50.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:50.857 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:50 smithi137 ceph-mon[35868]: Deploying daemon mds.g on smithi096 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:boot 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: fsmap 9 up:standby 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:50.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:50.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:50 smithi193 ceph-mon[36702]: Deploying daemon mds.g on smithi096 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:boot 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: fsmap 9 up:standby 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:50.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:50.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:50 smithi096 ceph-mon[31930]: Deploying daemon mds.g on smithi096 2024-03-18T15:37:50.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 -- 172.21.15.96:0/865203469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0658100700 msgr2=0x7f0658100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 --2- 172.21.15.96:0/865203469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0658100700 0x7f0658100b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0648009900 tx=0x7f064802bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 -- 172.21.15.96:0/865203469 shutdown_connections 2024-03-18T15:37:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 --2- 172.21.15.96:0/865203469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06581022b0 0x7f0658108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 --2- 172.21.15.96:0/865203469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581018f0 0x7f0658101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 --2- 172.21.15.96:0/865203469 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0658100700 0x7f0658100b20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 -- 172.21.15.96:0/865203469 >> 172.21.15.96:0/865203469 conn(0x7f06580fbc70 msgr2=0x7f06580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 -- 172.21.15.96:0/865203469 shutdown_connections 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.070+0000 7f065debf640 1 -- 172.21.15.96:0/865203469 wait complete. 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 Processor -- start 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 -- start start 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 0x7f0658197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581022b0 0x7f065819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f065810c370 con 0x7f06581018f0 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f065810c1f0 con 0x7f0658100700 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f065debf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f065810c4f0 con 0x7f06581022b0 2024-03-18T15:37:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42488/0 (socket says 172.21.15.96:42488) 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 -- 172.21.15.96:0/3583652405 learned_addr learned my addr 172.21.15.96:0/3583652405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f0657fff640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581022b0 0x7f065819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f06577fe640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 0x7f0658197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 -- 172.21.15.96:0/3583652405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581022b0 msgr2=0x7f065819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581022b0 0x7f065819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 -- 172.21.15.96:0/3583652405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 msgr2=0x7f0658197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 0x7f0658197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 -- 172.21.15.96:0/3583652405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065819f7b0 con 0x7f06581018f0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f06577fe640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 0x7f0658197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f064ffff640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f064000d770 tx=0x7f064000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.071+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640017070 con 0x7f06581018f0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.072+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0640005500 con 0x7f06581018f0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.072+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065819faa0 con 0x7f06581018f0 2024-03-18T15:37:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.072+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640004e00 con 0x7f06581018f0 2024-03-18T15:37:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.072+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065819ffa0 con 0x7f06581018f0 2024-03-18T15:37:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.072+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06581a01d0 con 0x7f06581018f0 2024-03-18T15:37:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.073+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f0640004fa0 con 0x7f06581018f0 2024-03-18T15:37:51.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.074+0000 7f06557fa640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.074+0000 7f06577fe640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.074+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f06400ce0b0 con 0x7f06581018f0 2024-03-18T15:37:51.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.074+0000 7f06577fe640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 0x7f06440a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0648036040 tx=0x7f0648000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.078+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f064009b5f0 con 0x7f06581018f0 2024-03-18T15:37:51.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.293+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06581a03a0 con 0x7f06581018f0 2024-03-18T15:37:51.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.294+0000 7f06557fa640 1 -- 172.21.15.96:0/3583652405 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7f064009aef0 con 0x7f06581018f0 2024-03-18T15:37:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:51.295 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T15:37:51.295 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 10 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 msgr2=0x7f06440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 0x7f06440a4cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0648036040 tx=0x7f0648000bc0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 msgr2=0x7f06581980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f064000d770 tx=0x7f064000dc30 comp rx=0 tx=0).stop 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 shutdown_connections 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06581022b0 0x7f065819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06581018f0 0x7f06581980d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 --2- 172.21.15.96:0/3583652405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0658100700 0x7f0658197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:51.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 >> 172.21.15.96:0/3583652405 conn(0x7f06580fbc70 msgr2=0x7f06580fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:51.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 shutdown_connections 2024-03-18T15:37:51.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:51.296+0000 7f065debf640 1 -- 172.21.15.96:0/3583652405 wait complete. 2024-03-18T15:37:51.613 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:51 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3583652405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:51.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:51 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3583652405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:51.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:51 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3583652405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:51.936 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:51 smithi096 systemd[1]: Starting Ceph mds.g for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:52.308 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 podman[73647]: 2024-03-18 15:37:51.922611935 +0000 UTC m=+0.017575393 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:52.308 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 podman[73647]: 2024-03-18 15:37:52.03798529 +0000 UTC m=+0.132948736 container create cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-18T15:37:52.308 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 podman[73647]: 2024-03-18 15:37:52.307894224 +0000 UTC m=+0.402857669 container init cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-18T15:37:52.686 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 podman[73647]: 2024-03-18 15:37:52.310629935 +0000 UTC m=+0.405593384 container start cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-18T15:37:52.686 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mds[73666]: -- 172.21.15.96:0/1355593414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a03111a40 con 0x556a030e4800 2024-03-18T15:37:52.686 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: starting mds.g at 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 bash[73647]: cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:37:52 smithi096 systemd[1]: Started Ceph mds.g for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:52.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:52 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:52.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:52 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:52 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:53.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:53.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 -- 172.21.15.96:0/787526217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201018f0 msgr2=0x7f8920101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:53.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 --2- 172.21.15.96:0/787526217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201018f0 0x7f8920101d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8910009900 tx=0x7f891002bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 -- 172.21.15.96:0/787526217 shutdown_connections 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 --2- 172.21.15.96:0/787526217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89201022b0 0x7f8920108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 --2- 172.21.15.96:0/787526217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201018f0 0x7f8920101d70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 --2- 172.21.15.96:0/787526217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8920100700 0x7f8920100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 -- 172.21.15.96:0/787526217 >> 172.21.15.96:0/787526217 conn(0x7f89200fbc70 msgr2=0x7f89200fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 -- 172.21.15.96:0/787526217 shutdown_connections 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.418+0000 7f8927f37640 1 -- 172.21.15.96:0/787526217 wait complete. 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 Processor -- start 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 -- start start 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8920100700 0x7f8920199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89201018f0 0x7f892019a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:53.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f892010c380 con 0x7f89201022b0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f892010c200 con 0x7f8920100700 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8927f37640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f892010c500 con 0x7f89201018f0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42532/0 (socket says 172.21.15.96:42532) 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 -- 172.21.15.96:0/4198607708 learned_addr learned my addr 172.21.15.96:0/4198607708 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89254d2640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89201018f0 0x7f892019a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f8925cd3640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8920100700 0x7f8920199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 -- 172.21.15.96:0/4198607708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89201018f0 msgr2=0x7f892019a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89201018f0 0x7f892019a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 -- 172.21.15.96:0/4198607708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8920100700 msgr2=0x7f8920199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8920100700 0x7f8920199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.419+0000 7f89264d4640 1 -- 172.21.15.96:0/4198607708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89201a1940 con 0x7f89201022b0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f89264d4640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f891800eaa0 tx=0x7f891800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:53.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891800cdb0 con 0x7f89201022b0 2024-03-18T15:37:53.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8918005020 con 0x7f89201022b0 2024-03-18T15:37:53.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89201a1c30 con 0x7f89201022b0 2024-03-18T15:37:53.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89180116b0 con 0x7f89201022b0 2024-03-18T15:37:53.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.420+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89201a2130 con 0x7f89201022b0 2024-03-18T15:37:53.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.421+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89201a2300 con 0x7f89201022b0 2024-03-18T15:37:53.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.422+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8918005560 con 0x7f89201022b0 2024-03-18T15:37:53.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.423+0000 7f8916ffd640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 0x7f890c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:53.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.423+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89180cec70 con 0x7f89201022b0 2024-03-18T15:37:53.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.423+0000 7f8925cd3640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 0x7f890c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:53.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.423+0000 7f8925cd3640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 0x7f890c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f892019b200 tx=0x7f891c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:53.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.426+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8918016030 con 0x7f89201022b0 2024-03-18T15:37:53.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.650+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89201a24d0 con 0x7f89201022b0 2024-03-18T15:37:53.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.651+0000 7f8916ffd640 1 -- 172.21.15.96:0/4198607708 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f891809c1b0 con 0x7f89201022b0 2024-03-18T15:37:53.652 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:53.652 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T15:37:53.652 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 11 2024-03-18T15:37:53.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 msgr2=0x7f890c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:53.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 0x7f890c0a4cc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f892019b200 tx=0x7f891c005f90 comp rx=0 tx=0).stop 2024-03-18T15:37:53.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 msgr2=0x7f89201a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:53.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f891800eaa0 tx=0x7f891800ef60 comp rx=0 tx=0).stop 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 shutdown_connections 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f890c0a2800 0x7f890c0a4cc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89201022b0 0x7f89201a12a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89201018f0 0x7f892019a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 --2- 172.21.15.96:0/4198607708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8920100700 0x7f8920199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 >> 172.21.15.96:0/4198607708 conn(0x7f89200fbc70 msgr2=0x7f89200fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 shutdown_connections 2024-03-18T15:37:53.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:53.653+0000 7f8927f37640 1 -- 172.21.15.96:0/4198607708 wait complete. 2024-03-18T15:37:53.841 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:53 smithi096 ceph-mon[31930]: Deploying daemon mds.a on smithi096 2024-03-18T15:37:53.841 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:53 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6838/36196340,v1:172.21.15.96:6839/36196340] up:boot 2024-03-18T15:37:53.841 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:53 smithi096 ceph-mon[31930]: fsmap 10 up:standby 2024-03-18T15:37:53.841 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:53 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:37:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:53 smithi137 ceph-mon[35868]: Deploying daemon mds.a on smithi096 2024-03-18T15:37:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:53 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6838/36196340,v1:172.21.15.96:6839/36196340] up:boot 2024-03-18T15:37:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:53 smithi137 ceph-mon[35868]: fsmap 10 up:standby 2024-03-18T15:37:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:53 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:37:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:53 smithi193 ceph-mon[36702]: Deploying daemon mds.a on smithi096 2024-03-18T15:37:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:53 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6838/36196340,v1:172.21.15.96:6839/36196340] up:boot 2024-03-18T15:37:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:53 smithi193 ceph-mon[36702]: fsmap 10 up:standby 2024-03-18T15:37:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:53 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:37:54.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:54 smithi137 ceph-mon[35868]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:54.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:54 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4198607708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:54 smithi096 ceph-mon[31930]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:54.859 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:54 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4198607708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:54.859 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:54 smithi096 systemd[1]: Starting Ceph mds.a for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:37:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:54 smithi193 ceph-mon[36702]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:54 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4198607708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:55.133 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:54 smithi096 podman[74003]: 2024-03-18 15:37:54.761696561 +0000 UTC m=+0.018226430 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:55.133 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:54 smithi096 podman[74003]: 2024-03-18 15:37:54.870509945 +0000 UTC m=+0.127039817 container create 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T15:37:55.133 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 podman[74003]: 2024-03-18 15:37:55.106912347 +0000 UTC m=+0.363442223 container init 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T15:37:55.133 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 podman[74003]: 2024-03-18 15:37:55.1095809 +0000 UTC m=+0.366110777 container start 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T15:37:55.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 ceph-mds[74022]: -- 172.21.15.96:0/832914285 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630134d7a40 con 0x5630134aac00 2024-03-18T15:37:55.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: starting mds.a at 2024-03-18T15:37:55.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 bash[74003]: 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 2024-03-18T15:37:55.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:37:55 smithi096 systemd[1]: Started Ceph mds.a for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:55.666 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:55.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 -- 172.21.15.96:0/191504173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 msgr2=0x7f43b8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 --2- 172.21.15.96:0/191504173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b8101df0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009900 tx=0x7f43ac02bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 -- 172.21.15.96:0/191504173 shutdown_connections 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 --2- 172.21.15.96:0/191504173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8102330 0x7f43b8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 --2- 172.21.15.96:0/191504173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b8101df0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 --2- 172.21.15.96:0/191504173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8100780 0x7f43b8100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 -- 172.21.15.96:0/191504173 >> 172.21.15.96:0/191504173 conn(0x7f43b80fbcf0 msgr2=0x7f43b80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 -- 172.21.15.96:0/191504173 shutdown_connections 2024-03-18T15:37:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.772+0000 7f43bd843640 1 -- 172.21.15.96:0/191504173 wait complete. 2024-03-18T15:37:55.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 Processor -- start 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 -- start start 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8100780 0x7f43b806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 0x7f43b819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810c370 con 0x7f43b8101970 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810c1f0 con 0x7f43b8102330 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43bd843640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43b810c4f0 con 0x7f43b8100780 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34944/0 (socket says 172.21.15.96:34944) 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 -- 172.21.15.96:0/1113447076 learned_addr learned my addr 172.21.15.96:0/1113447076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b6ffd640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8100780 0x7f43b806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b77fe640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 0x7f43b819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 -- 172.21.15.96:0/1113447076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8100780 msgr2=0x7f43b806aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8100780 0x7f43b806aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 -- 172.21.15.96:0/1113447076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 msgr2=0x7f43b819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 0x7f43b819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b67fc640 1 -- 172.21.15.96:0/1113447076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b819f740 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.773+0000 7f43b77fe640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 0x7f43b819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43b67fc640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009230 tx=0x7f43ac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac041070 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b819f9d0 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ac0044d0 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac034440 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.774+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b819fec0 con 0x7f43b8101970 2024-03-18T15:37:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.775+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b8107aa0 con 0x7f43b8101970 2024-03-18T15:37:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.776+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f43ac044070 con 0x7f43b8101970 2024-03-18T15:37:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.776+0000 7f43bc841640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 0x7f43a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.776+0000 7f43b6ffd640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 0x7f43a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.777+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43ac0f0220 con 0x7f43b8101970 2024-03-18T15:37:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.777+0000 7f43b6ffd640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 0x7f43a00a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43b81017d0 tx=0x7f43a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.781+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43ac0bd760 con 0x7f43b8101970 2024-03-18T15:37:55.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.992+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43b802d1b0 con 0x7f43b8101970 2024-03-18T15:37:55.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.992+0000 7f43bc841640 1 -- 172.21.15.96:0/1113447076 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f43ac0bd060 con 0x7f43b8101970 2024-03-18T15:37:55.993 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:55.993 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T15:37:55.993 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-03-18T15:37:55.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 msgr2=0x7f43a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:55.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 0x7f43a00a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43b81017d0 tx=0x7f43a8005f90 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 msgr2=0x7f43b819c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009230 tx=0x7f43ac005d50 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 shutdown_connections 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43a00a2800 0x7f43a00a4cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.994+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43b8102330 0x7f43b819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.995+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43b8101970 0x7f43b819c840 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.995+0000 7f43bd843640 1 --2- 172.21.15.96:0/1113447076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b8100780 0x7f43b806aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.995+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 >> 172.21.15.96:0/1113447076 conn(0x7f43b80fbcf0 msgr2=0x7f43b8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.995+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 shutdown_connections 2024-03-18T15:37:55.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:55.995+0000 7f43bd843640 1 -- 172.21.15.96:0/1113447076 wait complete. 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:56.535 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: Deploying daemon mds.b on smithi137 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6840/3021384858,v1:172.21.15.96:6841/3021384858] up:boot 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: fsmap 11 up:standby 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:37:56.536 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:56 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1113447076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:56.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: Deploying daemon mds.b on smithi137 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6840/3021384858,v1:172.21.15.96:6841/3021384858] up:boot 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: fsmap 11 up:standby 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:37:56.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:56 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1113447076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-18T15:37:56.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: Deploying daemon mds.b on smithi137 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6840/3021384858,v1:172.21.15.96:6841/3021384858] up:boot 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: fsmap 11 up:standby 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:37:56.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1113447076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:57.899 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:57 smithi137 podman[66305]: 2024-03-18 15:37:57.765733772 +0000 UTC m=+0.018193497 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:37:57.900 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:57 smithi137 podman[66305]: 2024-03-18 15:37:57.899301895 +0000 UTC m=+0.151761624 container create 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T15:37:58.008 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:37:58.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- 172.21.15.96:0/938361055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc000100700 msgr2=0x7fc000100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- 172.21.15.96:0/938361055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc000100700 0x7fc000100b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009900 tx=0x7fbffc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- 172.21.15.96:0/938361055 shutdown_connections 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- 172.21.15.96:0/938361055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 0x7fc000108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- 172.21.15.96:0/938361055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0001018f0 0x7fc000101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- 172.21.15.96:0/938361055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc000100700 0x7fc000100b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- 172.21.15.96:0/938361055 >> 172.21.15.96:0/938361055 conn(0x7fc0000fbc90 msgr2=0x7fc0000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- 172.21.15.96:0/938361055 shutdown_connections 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- 172.21.15.96:0/938361055 wait complete. 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 Processor -- start 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- start start 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc000100700 0x7fc000197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 0x7fc00019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:58.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010c350 con 0x7fc0001018f0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010c1d0 con 0x7fc0001022b0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.120+0000 7fc007e34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010c4d0 con 0x7fc000100700 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34962/0 (socket says 172.21.15.96:34962) 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 -- 172.21.15.96:0/1165956870 learned_addr learned my addr 172.21.15.96:0/1165956870 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc005bd0640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc000100700 0x7fc000197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0063d1640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 0x7fc00019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 -- 172.21.15.96:0/1165956870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc000100700 msgr2=0x7fc000197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc000100700 0x7fc000197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 -- 172.21.15.96:0/1165956870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 msgr2=0x7fc00019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 0x7fc00019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 -- 172.21.15.96:0/1165956870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00019f720 con 0x7fc0001018f0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc0053cf640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fc0001018f0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff0009c30 con 0x7fc0001018f0 2024-03-18T15:37:58.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff000f6b0 con 0x7fc0001018f0 2024-03-18T15:37:58.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00019fa10 con 0x7fc0001018f0 2024-03-18T15:37:58.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.121+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00019ff60 con 0x7fc0001018f0 2024-03-18T15:37:58.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.122+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0001a0190 con 0x7fc0001018f0 2024-03-18T15:37:58.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.124+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fbff00109d0 con 0x7fc0001018f0 2024-03-18T15:37:58.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.125+0000 7fbff6ffd640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 0x7fbfec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:37:58.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.125+0000 7fc005bd0640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 0x7fbfec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:37:58.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.125+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbff00ce120 con 0x7fc0001018f0 2024-03-18T15:37:58.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.125+0000 7fc005bd0640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 0x7fbfec0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:37:58.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.129+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fbff0009de0 con 0x7fc0001018f0 2024-03-18T15:37:58.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.343+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0001a0360 con 0x7fc0001018f0 2024-03-18T15:37:58.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.343+0000 7fbff6ffd640 1 -- 172.21.15.96:0/1165956870 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fbff009b010 con 0x7fc0001018f0 2024-03-18T15:37:58.343 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:37:58.343 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T15:37:58.344 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 msgr2=0x7fbfec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 0x7fbfec0a4cc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbffc000c00 tx=0x7fbffc005fd0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 msgr2=0x7fc000198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).stop 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 shutdown_connections 2024-03-18T15:37:58.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fbfec0a2800 0x7fbfec0a4cc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0001022b0 0x7fc00019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0001018f0 0x7fc000198040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 --2- 172.21.15.96:0/1165956870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc000100700 0x7fc000197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 >> 172.21.15.96:0/1165956870 conn(0x7fc0000fbc90 msgr2=0x7fc000069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 shutdown_connections 2024-03-18T15:37:58.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:37:58.345+0000 7fc007e34640 1 -- 172.21.15.96:0/1165956870 wait complete. 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 podman[66305]: 2024-03-18 15:37:58.17646667 +0000 UTC m=+0.428926379 container init 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 podman[66305]: 2024-03-18 15:37:58.1793362 +0000 UTC m=+0.431795902 container start 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mds[66323]: -- 172.21.15.137:0/594525647 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564120241a40 con 0x564120214c00 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: starting mds.b at 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 bash[66305]: 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 2024-03-18T15:37:58.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:37:58 smithi137 systemd[1]: Started Ceph mds.b for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1165956870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:58 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1165956870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:58 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1165956870' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:58 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:37:59.547 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:59 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:boot 2024-03-18T15:37:59.547 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:59 smithi096 ceph-mon[31930]: fsmap 12 up:standby 2024-03-18T15:37:59.547 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:37:59.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:37:59 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:59 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:boot 2024-03-18T15:37:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:59 smithi137 ceph-mon[35868]: fsmap 12 up:standby 2024-03-18T15:37:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:37:59.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:37:59 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:37:59.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:59 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:boot 2024-03-18T15:37:59.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:59 smithi193 ceph-mon[36702]: fsmap 12 up:standby 2024-03-18T15:37:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:37:59.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:37:59 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:38:00.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- 172.21.15.96:0/2395469832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4102360 msgr2=0x7fd0a4104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- 172.21.15.96:0/2395469832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4102360 0x7fd0a4104750 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a002bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- 172.21.15.96:0/2395469832 shutdown_connections 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- 172.21.15.96:0/2395469832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a41075c0 0x7fd0a41099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- 172.21.15.96:0/2395469832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4104c90 0x7fd0a4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- 172.21.15.96:0/2395469832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a4102360 0x7fd0a4104750 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- 172.21.15.96:0/2395469832 >> 172.21.15.96:0/2395469832 conn(0x7fd0a40fbd10 msgr2=0x7fd0a40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- 172.21.15.96:0/2395469832 shutdown_connections 2024-03-18T15:38:00.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- 172.21.15.96:0/2395469832 wait complete. 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 Processor -- start 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- start start 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4102360 0x7fd0a4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a4104c90 0x7fd0a41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410d500 con 0x7fd0a41075c0 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410d380 con 0x7fd0a4104c90 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.462+0000 7fd0ac109640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a410d680 con 0x7fd0a4102360 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34990/0 (socket says 172.21.15.96:34990) 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 -- 172.21.15.96:0/4043201222 learned_addr learned my addr 172.21.15.96:0/4043201222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0a9ea5640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4102360 0x7fd0a4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 -- 172.21.15.96:0/4043201222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4102360 msgr2=0x7fd0a4197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4102360 0x7fd0a4197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 -- 172.21.15.96:0/4043201222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a4104c90 msgr2=0x7fd0a41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a4104c90 0x7fd0a41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 -- 172.21.15.96:0/4043201222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a419f7d0 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0aa6a6640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00eaa0 tx=0x7fd09c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c00cdb0 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a419fac0 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c005020 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0116b0 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.463+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a41a0010 con 0x7fd0a41075c0 2024-03-18T15:38:00.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.464+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a40fd800 con 0x7fd0a41075c0 2024-03-18T15:38:00.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.465+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd09c005560 con 0x7fd0a41075c0 2024-03-18T15:38:00.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.466+0000 7fd09affd640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 0x7fd0900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.466+0000 7fd0a9ea5640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 0x7fd0900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.466+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd09c0ce830 con 0x7fd0a41075c0 2024-03-18T15:38:00.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.466+0000 7fd0a9ea5640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 0x7fd0900a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002c7b0 tx=0x7fd0a0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:00.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.470+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd09c016030 con 0x7fd0a41075c0 2024-03-18T15:38:00.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.684+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a402d590 con 0x7fd0a41075c0 2024-03-18T15:38:00.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.685+0000 7fd09affd640 1 -- 172.21.15.96:0/4043201222 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fd0a402d590 con 0x7fd0a41075c0 2024-03-18T15:38:00.686 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:00.686 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24460,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-03-18T15:38:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 13 2024-03-18T15:38:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:00 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 msgr2=0x7fd0900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 0x7fd0900a4cc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0a002c7b0 tx=0x7fd0a0005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 msgr2=0x7fd0a419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00eaa0 tx=0x7fd09c00ef60 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 shutdown_connections 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd0900a2800 0x7fd0900a4cc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0a41075c0 0x7fd0a419f160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd0a4104c90 0x7fd0a41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 --2- 172.21.15.96:0/4043201222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a4102360 0x7fd0a4197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 >> 172.21.15.96:0/4043201222 conn(0x7fd0a40fbd10 msgr2=0x7fd0a4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 shutdown_connections 2024-03-18T15:38:00.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.687+0000 7fd0ac109640 1 -- 172.21.15.96:0/4043201222 wait complete. 2024-03-18T15:38:00.699 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 -- 172.21.15.96:0/2051015382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a01029f0 msgr2=0x7f81a0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 --2- 172.21.15.96:0/2051015382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a01029f0 0x7f81a0102e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8188009900 tx=0x7f818802bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 -- 172.21.15.96:0/2051015382 shutdown_connections 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 --2- 172.21.15.96:0/2051015382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01033b0 0x7f81a0109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 --2- 172.21.15.96:0/2051015382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a01029f0 0x7f81a0102e70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 --2- 172.21.15.96:0/2051015382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a0101800 0x7f81a0101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 -- 172.21.15.96:0/2051015382 >> 172.21.15.96:0/2051015382 conn(0x7f81a00fcd90 msgr2=0x7f81a00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 -- 172.21.15.96:0/2051015382 shutdown_connections 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.800+0000 7f81a5612640 1 -- 172.21.15.96:0/2051015382 wait complete. 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 Processor -- start 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 -- start start 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 0x7f81a0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a01033b0 0x7f81a019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010d480 con 0x7f81a0101800 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010d300 con 0x7f81a01033b0 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81a5612640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010d600 con 0x7f81a01029f0 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f819effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f819effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35008/0 (socket says 172.21.15.96:35008) 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f819effd640 1 -- 172.21.15.96:0/2071472709 learned_addr learned my addr 172.21.15.96:0/2071472709 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f81967fc640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 0x7f81a0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.801+0000 7f819f7fe640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a01033b0 0x7f81a019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 -- 172.21.15.96:0/2071472709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 msgr2=0x7f81a0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 0x7f81a0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 -- 172.21.15.96:0/2071472709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a01033b0 msgr2=0x7f81a019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a01033b0 0x7f81a019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 -- 172.21.15.96:0/2071472709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a019f720 con 0x7f81a0101800 2024-03-18T15:38:00.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f81967fc640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 0x7f81a0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:00.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.802+0000 7f819effd640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f819000d950 tx=0x7f819000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:00.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.804+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190017020 con 0x7f81a0101800 2024-03-18T15:38:00.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.804+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a019fa10 con 0x7f81a0101800 2024-03-18T15:38:00.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.804+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190009c30 con 0x7f81a0101800 2024-03-18T15:38:00.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.804+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819000f740 con 0x7f81a0101800 2024-03-18T15:38:00.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.804+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a019ff80 con 0x7f81a0101800 2024-03-18T15:38:00.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.805+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a0068a20 con 0x7f81a0101800 2024-03-18T15:38:00.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.806+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f819001a020 con 0x7f81a0101800 2024-03-18T15:38:00.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.807+0000 7f819cff9640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 0x7f818c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:00.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.807+0000 7f81967fc640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 0x7f818c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:00.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.807+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81900cdf00 con 0x7f81a0101800 2024-03-18T15:38:00.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.807+0000 7f81967fc640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 0x7f818c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f818802c7b0 tx=0x7f8188005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:00.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:00.811+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8190021070 con 0x7f81a0101800 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.848 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:00 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:00 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.019+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81a002d1b0 con 0x7f81a0101800 2024-03-18T15:38:01.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.020+0000 7f819cff9640 1 -- 172.21.15.96:0/2071472709 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18408 (secure 0 0 0) 0x7f819009b440 con 0x7f81a0101800 2024-03-18T15:38:01.021 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:01.021 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":82,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:37:13.033553+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 msgr2=0x7f818c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 0x7f818c0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f818802c7b0 tx=0x7f8188005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 msgr2=0x7f81a0197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f819000d950 tx=0x7f819000de10 comp rx=0 tx=0).stop 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 shutdown_connections 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f818c0a2800 0x7f818c0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81a01033b0 0x7f81a019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81a01029f0 0x7f81a0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 --2- 172.21.15.96:0/2071472709 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a0101800 0x7f81a0197b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 >> 172.21.15.96:0/2071472709 conn(0x7f81a00fcd90 msgr2=0x7f81a0105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 shutdown_connections 2024-03-18T15:38:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.022+0000 7f81a5612640 1 -- 172.21.15.96:0/2071472709 wait complete. 2024-03-18T15:38:01.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:38:01.035 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:38:01.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- 172.21.15.96:0/2890644101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56781006e0 msgr2=0x7f5678100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- 172.21.15.96:0/2890644101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56781006e0 0x7f5678100b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5674009900 tx=0x7f567402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- 172.21.15.96:0/2890644101 shutdown_connections 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- 172.21.15.96:0/2890644101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102290 0x7f5678108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- 172.21.15.96:0/2890644101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- 172.21.15.96:0/2890644101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56781006e0 0x7f5678100b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- 172.21.15.96:0/2890644101 >> 172.21.15.96:0/2890644101 conn(0x7f56780fbc70 msgr2=0x7f56780fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- 172.21.15.96:0/2890644101 shutdown_connections 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- 172.21.15.96:0/2890644101 wait complete. 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 Processor -- start 2024-03-18T15:38:01.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- start start 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 0x7f5678196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c360 con 0x7f5678102290 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c1e0 con 0x7f56781018d0 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.141+0000 7f567f612640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f567810c4e0 con 0x7f56781006e0 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567cbad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567d3ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 0x7f5678196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35040/0 (socket says 172.21.15.96:35040) 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 -- 172.21.15.96:0/2046975703 learned_addr learned my addr 172.21.15.96:0/2046975703 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 -- 172.21.15.96:0/2046975703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 msgr2=0x7f5678196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 0x7f5678196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 -- 172.21.15.96:0/2046975703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 msgr2=0x7f5678196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 -- 172.21.15.96:0/2046975703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567819e640 con 0x7f5678102290 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567cbad640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567d3ae640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 0x7f5678196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567dbaf640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f567000eb10 tx=0x7f567000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004b20 con 0x7f5678102290 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5670004cc0 con 0x7f5678102290 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567819e930 con 0x7f5678102290 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670010760 con 0x7f5678102290 2024-03-18T15:38:01.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.142+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567819ee80 con 0x7f5678102290 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.143+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678107ad0 con 0x7f5678102290 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.145+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f5670005000 con 0x7f5678102290 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.146+0000 7f566e7fc640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 0x7f56640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.146+0000 7f567d3ae640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 0x7f56640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.146+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56700cf010 con 0x7f5678102290 2024-03-18T15:38:01.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.146+0000 7f567d3ae640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 0x7f56640a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5674000c00 tx=0x7f5674005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.150+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f567009c550 con 0x7f5678102290 2024-03-18T15:38:01.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.328+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56780623e0 con 0x7f5678102290 2024-03-18T15:38:01.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.328+0000 7f566e7fc640 1 -- 172.21.15.96:0/2046975703 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f567009be50 con 0x7f5678102290 2024-03-18T15:38:01.329 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:38:01.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 msgr2=0x7f56640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 0x7f56640a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5674000c00 tx=0x7f5674005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 msgr2=0x7f567819dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f567000eb10 tx=0x7f567000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 shutdown_connections 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f56640a2800 0x7f56640a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5678102290 0x7f567819dfd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56781018d0 0x7f5678196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 --2- 172.21.15.96:0/2046975703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781006e0 0x7f5678196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 >> 172.21.15.96:0/2046975703 conn(0x7f56780fbc70 msgr2=0x7f5678104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 shutdown_connections 2024-03-18T15:38:01.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.330+0000 7f567f612640 1 -- 172.21.15.96:0/2046975703 wait complete. 2024-03-18T15:38:01.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T15:38:01.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 -- 172.21.15.96:0/1728516939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a0100700 msgr2=0x7f82a0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 --2- 172.21.15.96:0/1728516939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a0100700 0x7f82a0100b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8290009900 tx=0x7f829002bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:01.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 -- 172.21.15.96:0/1728516939 shutdown_connections 2024-03-18T15:38:01.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 --2- 172.21.15.96:0/1728516939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a01022b0 0x7f82a0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 --2- 172.21.15.96:0/1728516939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 0x7f82a0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 --2- 172.21.15.96:0/1728516939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a0100700 0x7f82a0100b20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 -- 172.21.15.96:0/1728516939 >> 172.21.15.96:0/1728516939 conn(0x7f82a00fbc70 msgr2=0x7f82a00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 -- 172.21.15.96:0/1728516939 shutdown_connections 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 -- 172.21.15.96:0/1728516939 wait complete. 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.451+0000 7f82a58b2640 1 Processor -- start 2024-03-18T15:38:01.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 -- start start 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a0100700 0x7f82a0104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 0x7f82a0102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010c350 con 0x7f82a01022b0 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010c1d0 con 0x7f82a01018f0 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f82a58b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82a010c4d0 con 0x7f82a0100700 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35058/0 (socket says 172.21.15.96:35058) 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a0100700 0x7f82a0104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 -- 172.21.15.96:0/2745746321 learned_addr learned my addr 172.21.15.96:0/2745746321 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829e7fc640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 0x7f82a0102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 -- 172.21.15.96:0/2745746321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a0100700 msgr2=0x7f82a0104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a0100700 0x7f82a0104640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 -- 172.21.15.96:0/2745746321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 msgr2=0x7f82a0102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 0x7f82a0102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 -- 172.21.15.96:0/2745746321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a0103ad0 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.452+0000 7f829f7fe640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8298009fd0 tx=0x7f829800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.453+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298017070 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.453+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8298003e50 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.453+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a01a1d00 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.453+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82980106c0 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.453+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a01a21d0 con 0x7f82a01022b0 2024-03-18T15:38:01.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.454+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a01a23a0 con 0x7f82a01022b0 2024-03-18T15:38:01.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.455+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f8298003970 con 0x7f82a01022b0 2024-03-18T15:38:01.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.456+0000 7f82a48b0640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 0x7f82880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.456+0000 7f829effd640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 0x7f82880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.456+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82980ce270 con 0x7f82a01022b0 2024-03-18T15:38:01.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.457+0000 7f829effd640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 0x7f82880a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8290000c00 tx=0x7f8290005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.461+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f829809b160 con 0x7f82a01022b0 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.648+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f82a01a2730 con 0x7f82a01022b0 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.648+0000 7f82a48b0640 1 -- 172.21.15.96:0/2745746321 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f829809f200 con 0x7f82a01022b0 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-03-18T15:38:01.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:38:01.650 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 msgr2=0x7f82880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 0x7f82880a4cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8290000c00 tx=0x7f8290005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 msgr2=0x7f82a0103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8298009fd0 tx=0x7f829800b260 comp rx=0 tx=0).stop 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 shutdown_connections 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82880a2800 0x7f82880a4cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82a01022b0 0x7f82a0103210 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82a01018f0 0x7f82a0102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 --2- 172.21.15.96:0/2745746321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a0100700 0x7f82a0104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 >> 172.21.15.96:0/2745746321 conn(0x7f82a00fbc70 msgr2=0x7f82a00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 shutdown_connections 2024-03-18T15:38:01.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.650+0000 7f82a58b2640 1 -- 172.21.15.96:0/2745746321 wait complete. 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4043201222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2071472709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2046975703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:38:01.661 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.662 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T15:38:01.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.766+0000 7fa5cb387640 1 -- 172.21.15.96:0/2403360591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c4102290 msgr2=0x7fa5c4108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.766+0000 7fa5cb387640 1 --2- 172.21.15.96:0/2403360591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c4102290 0x7fa5c4108b20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ca0 tx=0x7fa5bc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- 172.21.15.96:0/2403360591 shutdown_connections 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- 172.21.15.96:0/2403360591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c4102290 0x7fa5c4108b20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- 172.21.15.96:0/2403360591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41018d0 0x7fa5c4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- 172.21.15.96:0/2403360591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c41006e0 0x7fa5c4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- 172.21.15.96:0/2403360591 >> 172.21.15.96:0/2403360591 conn(0x7fa5c40fbc70 msgr2=0x7fa5c40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:01.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- 172.21.15.96:0/2403360591 shutdown_connections 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- 172.21.15.96:0/2403360591 wait complete. 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 Processor -- start 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- start start 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41006e0 0x7fa5c4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c4102290 0x7fa5c419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d480 con 0x7fa5c41018d0 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d300 con 0x7fa5c4102290 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5cb387640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c410d600 con 0x7fa5c41006e0 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5c8922640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5c8922640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35076/0 (socket says 172.21.15.96:35076) 2024-03-18T15:38:01.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5c8922640 1 -- 172.21.15.96:0/642748830 learned_addr learned my addr 172.21.15.96:0/642748830 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5c9924640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c4102290 0x7fa5c419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.767+0000 7fa5c9123640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41006e0 0x7fa5c4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 -- 172.21.15.96:0/642748830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41006e0 msgr2=0x7fa5c4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41006e0 0x7fa5c4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 -- 172.21.15.96:0/642748830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c4102290 msgr2=0x7fa5c419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c4102290 0x7fa5c419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 -- 172.21.15.96:0/642748830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419f6d0 con 0x7fa5c41018d0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5c8922640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4017070 con 0x7fa5c41018d0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419f9c0 con 0x7fa5c41018d0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4009c30 con 0x7fa5c41018d0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400f6b0 con 0x7fa5c41018d0 2024-03-18T15:38:01.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.768+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c419ff10 con 0x7fa5c41018d0 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4043201222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2071472709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2046975703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:38:01.772 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.769+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4107a00 con 0x7fa5c41018d0 2024-03-18T15:38:01.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.770+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fa5b40109d0 con 0x7fa5c41018d0 2024-03-18T15:38:01.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.770+0000 7fa5ba7fc640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 0x7fa5b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:01.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.771+0000 7fa5c9123640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 0x7fa5b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:01.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.771+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5b40ce180 con 0x7fa5c41018d0 2024-03-18T15:38:01.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.771+0000 7fa5c9123640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 0x7fa5b00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4101730 tx=0x7fa5c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4043201222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2071472709' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2046975703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:38:01.777 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:01.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.777+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa5b4009de0 con 0x7fa5c41018d0 2024-03-18T15:38:01.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:01.963+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa5c402d3f0 con 0x7fa5c41018d0 2024-03-18T15:38:02.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.805+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/642748830 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fa5b409b070 con 0x7fa5c41018d0 2024-03-18T15:38:02.806 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 msgr2=0x7fa5b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 0x7fa5b00a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4101730 tx=0x7fa5c000b390 comp rx=0 tx=0).stop 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 msgr2=0x7fa5c4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).stop 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 shutdown_connections 2024-03-18T15:38:02.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5b00a2800 0x7fa5b00a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5c4102290 0x7fa5c419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5c41018d0 0x7fa5c4197ff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 --2- 172.21.15.96:0/642748830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5c41006e0 0x7fa5c4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 >> 172.21.15.96:0/642748830 conn(0x7fa5c40fbc70 msgr2=0x7fa5c4104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 shutdown_connections 2024-03-18T15:38:02.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.807+0000 7fa5cb387640 1 -- 172.21.15.96:0/642748830 wait complete. 2024-03-18T15:38:02.820 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:02 smithi096 ceph-mon[31930]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:02.821 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:02 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2745746321' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T15:38:02.821 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:02 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T15:38:02.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T15:38:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:02 smithi137 ceph-mon[35868]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:02 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2745746321' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T15:38:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:02 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T15:38:02.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:02 smithi193 ceph-mon[36702]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:02 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2745746321' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-18T15:38:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:02 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2427498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008104ba0 msgr2=0x7fe008106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2427498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008104ba0 0x7fe008106f90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009900 tx=0x7fdff402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2427498924 shutdown_connections 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2427498924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081074d0 0x7fe008109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2427498924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008104ba0 0x7fe008106f90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2427498924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe008102270 0x7fe008104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2427498924 >> 172.21.15.96:0/2427498924 conn(0x7fe0080fbc80 msgr2=0x7fe0080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2427498924 shutdown_connections 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2427498924 wait complete. 2024-03-18T15:38:02.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 Processor -- start 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.929+0000 7fe00ebd3640 1 -- start start 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008104ba0 0x7fe008198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0081074d0 0x7fe00819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe00810d540 con 0x7fe008102270 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe00810d3c0 con 0x7fe0081074d0 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe00810d6c0 con 0x7fe008104ba0 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35102/0 (socket says 172.21.15.96:35102) 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 -- 172.21.15.96:0/2808649244 learned_addr learned my addr 172.21.15.96:0/2808649244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe007fff640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008104ba0 0x7fe008198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00d170640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0081074d0 0x7fe00819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 -- 172.21.15.96:0/2808649244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008104ba0 msgr2=0x7fe008198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:02.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008104ba0 0x7fe008198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 -- 172.21.15.96:0/2808649244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0081074d0 msgr2=0x7fe00819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0081074d0 0x7fe00819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 -- 172.21.15.96:0/2808649244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00819f810 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00c96f640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b4a0 tx=0x7fdffc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc014070 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdffc010af0 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00819fb00 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdffc017d10 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.930+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a0020 con 0x7fe008102270 2024-03-18T15:38:02.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.931+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0080fd770 con 0x7fe008102270 2024-03-18T15:38:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.932+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fdffc017070 con 0x7fe008102270 2024-03-18T15:38:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.933+0000 7fe005ffb640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 0x7fdff80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.933+0000 7fe007fff640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 0x7fdff80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.933+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fdffc0cda60 con 0x7fe008102270 2024-03-18T15:38:02.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.933+0000 7fe007fff640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 0x7fdff80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdff402c170 tx=0x7fdff402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:02.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:02.938+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdffc09ae30 con 0x7fe008102270 2024-03-18T15:38:03.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.114+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe008062480 con 0x7fe008102270 2024-03-18T15:38:03.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.810+0000 7fe005ffb640 1 -- 172.21.15.96:0/2808649244 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fdffc09a730 con 0x7fe008102270 2024-03-18T15:38:03.811 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-03-18T15:38:03.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 msgr2=0x7fdff80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:03.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 0x7fdff80a4cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdff402c170 tx=0x7fdff402df90 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 msgr2=0x7fe008197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00b4a0 tx=0x7fdffc00b960 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 shutdown_connections 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdff80a2800 0x7fdff80a4cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0081074d0 0x7fe00819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008104ba0 0x7fe008198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 --2- 172.21.15.96:0/2808649244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe008102270 0x7fe008197bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 >> 172.21.15.96:0/2808649244 conn(0x7fe0080fbc80 msgr2=0x7fe008102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 shutdown_connections 2024-03-18T15:38:03.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.813+0000 7fe00ebd3640 1 -- 172.21.15.96:0/2808649244 wait complete. 2024-03-18T15:38:03.834 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T15:38:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T15:38:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T15:38:03.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:03 smithi096 ceph-mon[31930]: pgmap v284: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:03.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 -- 172.21.15.96:0/3957658770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 msgr2=0x7fb394101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:03.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 --2- 172.21.15.96:0/3957658770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394101c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb390009900 tx=0x7fb39002bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:03.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 -- 172.21.15.96:0/3957658770 shutdown_connections 2024-03-18T15:38:03.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 --2- 172.21.15.96:0/3957658770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 0x7fb394109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 --2- 172.21.15.96:0/3957658770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 0x7fb394102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 --2- 172.21.15.96:0/3957658770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394101c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 -- 172.21.15.96:0/3957658770 >> 172.21.15.96:0/3957658770 conn(0x7fb3940fcd40 msgr2=0x7fb3940ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 -- 172.21.15.96:0/3957658770 shutdown_connections 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.939+0000 7fb39b777640 1 -- 172.21.15.96:0/3957658770 wait complete. 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 Processor -- start 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 -- start start 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 0x7fb394198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 0x7fb39419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410d440 con 0x7fb3941017f0 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410d2c0 con 0x7fb3941029e0 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb39b777640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb39410d5c0 con 0x7fb3941033a0 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399513640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:03.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399513640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55452/0 (socket says 172.21.15.96:55452) 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399513640 1 -- 172.21.15.96:0/3719850475 learned_addr learned my addr 172.21.15.96:0/3719850475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb398d12640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 0x7fb394198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399d14640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 0x7fb39419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399513640 1 -- 172.21.15.96:0/3719850475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 msgr2=0x7fb39419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.940+0000 7fb399513640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 0x7fb39419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb399513640 1 -- 172.21.15.96:0/3719850475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 msgr2=0x7fb394198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb399513640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 0x7fb394198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb399513640 1 -- 172.21.15.96:0/3719850475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39419f720 con 0x7fb3941017f0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb399513640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb390005ab0 tx=0x7fb390005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390041070 con 0x7fb3941017f0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39419f9b0 con 0x7fb3941017f0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3900054e0 con 0x7fb3941017f0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390034440 con 0x7fb3941017f0 2024-03-18T15:38:03.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.941+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb39419fef0 con 0x7fb3941017f0 2024-03-18T15:38:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.942+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb394108be0 con 0x7fb3941017f0 2024-03-18T15:38:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.943+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb390044070 con 0x7fb3941017f0 2024-03-18T15:38:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.944+0000 7fb38a7fc640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 0x7fb3800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:03.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.944+0000 7fb398d12640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 0x7fb3800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:03.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.944+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fb3900f0470 con 0x7fb3941017f0 2024-03-18T15:38:03.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.944+0000 7fb398d12640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 0x7fb3800a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb394199010 tx=0x7fb38400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:03.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:03.949+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3900bd070 con 0x7fb3941017f0 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T15:38:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T15:38:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T15:38:04.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:03 smithi137 ceph-mon[35868]: pgmap v284: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:04.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.166+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb39402d1b0 con 0x7fb3941017f0 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:38:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:38:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:38:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/642748830' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-18T15:38:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: osdmap e83: 12 total, 12 up, 12 in 2024-03-18T15:38:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-18T15:38:04.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:03 smithi193 ceph-mon[36702]: pgmap v284: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:38:04.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.811+0000 7fb38a7fc640 1 -- 172.21.15.96:0/3719850475 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb3900c1110 con 0x7fb3941017f0 2024-03-18T15:38:04.813 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 msgr2=0x7fb3800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 0x7fb3800a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb394199010 tx=0x7fb38400b390 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 msgr2=0x7fb394197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb390005ab0 tx=0x7fb390005dc0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 shutdown_connections 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3800a2800 0x7fb3800a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3941033a0 0x7fb39419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3941029e0 0x7fb394198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 --2- 172.21.15.96:0/3719850475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3941017f0 0x7fb394197b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 >> 172.21.15.96:0/3719850475 conn(0x7fb3940fcd40 msgr2=0x7fb394105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 shutdown_connections 2024-03-18T15:38:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.814+0000 7fb39b777640 1 -- 172.21.15.96:0/3719850475 wait complete. 2024-03-18T15:38:04.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:04 smithi096 ceph-mon[31930]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T15:38:04.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T15:38:04.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:04 smithi096 ceph-mon[31930]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T15:38:04.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T15:38:04.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:04 smithi096 ceph-mon[31930]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-18T15:38:04.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:38:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/2185810460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 msgr2=0x7f9f98109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/2185810460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f98109c30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009ca0 tx=0x7f9f9002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/2185810460 shutdown_connections 2024-03-18T15:38:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/2185810460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f98109c30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/2185810460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 0x7f9f98102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/2185810460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 0x7f9f98101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/2185810460 >> 172.21.15.96:0/2185810460 conn(0x7f9f980fcdc0 msgr2=0x7f9f980ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/2185810460 shutdown_connections 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.929+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/2185810460 wait complete. 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 Processor -- start 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 -- start start 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 0x7f9f98197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 0x7f9f98198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9810d560 con 0x7f9f981033a0 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9810d3e0 con 0x7f9f981029e0 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9cbd3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9810d6e0 con 0x7f9f981017f0 2024-03-18T15:38:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55482/0 (socket says 172.21.15.96:55482) 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 -- 172.21.15.96:0/659490690 learned_addr learned my addr 172.21.15.96:0/659490690 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f9659c640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 0x7f9f98197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f95d9b640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 0x7f9f98198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 -- 172.21.15.96:0/659490690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 msgr2=0x7f9f98197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 0x7f9f98197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 -- 172.21.15.96:0/659490690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 msgr2=0x7f9f98198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 0x7f9f98198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 -- 172.21.15.96:0/659490690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9819f7f0 con 0x7f9f981033a0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.930+0000 7f9f96d9d640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f90037040 tx=0x7f9f90032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.931+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90039040 con 0x7f9f981033a0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.931+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9819fa80 con 0x7f9f981033a0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.931+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f90044070 con 0x7f9f981033a0 2024-03-18T15:38:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.931+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90004330 con 0x7f9f981033a0 2024-03-18T15:38:04.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.931+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a0040 con 0x7f9f981033a0 2024-03-18T15:38:04.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.932+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98108b10 con 0x7f9f981033a0 2024-03-18T15:38:04.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.933+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f9f90003a00 con 0x7f9f981033a0 2024-03-18T15:38:04.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.933+0000 7f9f877fe640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 0x7f9f800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:04.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.933+0000 7f9f9659c640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 0x7f9f800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:04.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.933+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9f900f1200 con 0x7f9f981033a0 2024-03-18T15:38:04.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.934+0000 7f9f9659c640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 0x7f9f800a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f98102840 tx=0x7f9f88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:04.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:04.938+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9f900be3f0 con 0x7f9f981033a0 2024-03-18T15:38:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:04 smithi137 ceph-mon[35868]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T15:38:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T15:38:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:04 smithi137 ceph-mon[35868]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T15:38:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T15:38:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:04 smithi137 ceph-mon[35868]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-18T15:38:05.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.113+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f980623e0 con 0x7f9f981033a0 2024-03-18T15:38:05.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.115+0000 7f9f877fe640 1 -- 172.21.15.96:0/659490690 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22184 (secure 0 0 0) 0x7f9f900bdcf0 con 0x7f9f981033a0 2024-03-18T15:38:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:05.116 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":85,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:38:04.167494+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 msgr2=0x7f9f800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 0x7f9f800a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f98102840 tx=0x7f9f88008040 comp rx=0 tx=0).stop 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 msgr2=0x7f9f9819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9f90037040 tx=0x7f9f90032f90 comp rx=0 tx=0).stop 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 shutdown_connections 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9f800a2800 0x7f9f800a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f981033a0 0x7f9f9819f180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f981029e0 0x7f9f98198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 --2- 172.21.15.96:0/659490690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f981017f0 0x7f9f98197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 >> 172.21.15.96:0/659490690 conn(0x7f9f980fcdc0 msgr2=0x7f9f98105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 shutdown_connections 2024-03-18T15:38:05.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.117+0000 7f9f9cbd3640 1 -- 172.21.15.96:0/659490690 wait complete. 2024-03-18T15:38:05.129 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:38:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:04 smithi193 ceph-mon[36702]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-18T15:38:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2808649244' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-18T15:38:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:04 smithi193 ceph-mon[36702]: osdmap e84: 12 total, 12 up, 12 in 2024-03-18T15:38:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-18T15:38:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:04 smithi193 ceph-mon[36702]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-18T15:38:05.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- 172.21.15.96:0/2980093420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 msgr2=0x7fb758101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/2980093420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758101c20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb748009900 tx=0x7fb74802bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- 172.21.15.96:0/2980093420 shutdown_connections 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/2980093420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7581033b0 0x7fb758109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/2980093420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 0x7fb758102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/2980093420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758101c20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- 172.21.15.96:0/2980093420 >> 172.21.15.96:0/2980093420 conn(0x7fb7580fcd90 msgr2=0x7fb7580ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- 172.21.15.96:0/2980093420 shutdown_connections 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- 172.21.15.96:0/2980093420 wait complete. 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 Processor -- start 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 -- start start 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.230+0000 7fb75f2ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 0x7fb758198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7581033b0 0x7fb75819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d4d0 con 0x7fb758101800 2024-03-18T15:38:05.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d350 con 0x7fb7581033b0 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d650 con 0x7fb7581029f0 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55490/0 (socket says 172.21.15.96:55490) 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 -- 172.21.15.96:0/3700403401 learned_addr learned my addr 172.21.15.96:0/3700403401 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75c847640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 0x7fb758198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 -- 172.21.15.96:0/3700403401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 msgr2=0x7fb758198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 0x7fb758198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 -- 172.21.15.96:0/3700403401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7581033b0 msgr2=0x7fb75819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7581033b0 0x7fb75819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 -- 172.21.15.96:0/3700403401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819f730 con 0x7fb758101800 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75d048640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb748000c00 tx=0x7fb748004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748041070 con 0x7fb758101800 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819f9c0 con 0x7fb758101800 2024-03-18T15:38:05.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7480054e0 con 0x7fb758101800 2024-03-18T15:38:05.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7480346a0 con 0x7fb758101800 2024-03-18T15:38:05.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.231+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75819feb0 con 0x7fb758101800 2024-03-18T15:38:05.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.232+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758108ac0 con 0x7fb758101800 2024-03-18T15:38:05.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.233+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fb748044070 con 0x7fb758101800 2024-03-18T15:38:05.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.234+0000 7fb74e7fc640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 0x7fb7440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.234+0000 7fb75c847640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 0x7fb7440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.234+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fb7480f07b0 con 0x7fb758101800 2024-03-18T15:38:05.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.234+0000 7fb75c847640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 0x7fb7440a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb754005e10 tx=0x7fb754005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.238+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb7480bd9a0 con 0x7fb758101800 2024-03-18T15:38:05.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.414+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb75802d440 con 0x7fb758101800 2024-03-18T15:38:05.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.416+0000 7fb74e7fc640 1 -- 172.21.15.96:0/3700403401 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22184 (secure 0 0 0) 0x7fb748003e20 con 0x7fb758101800 2024-03-18T15:38:05.417 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:05.417 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":85,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:38:04.167494+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.417+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 msgr2=0x7fb7440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.417+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 0x7fb7440a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb754005e10 tx=0x7fb754005da0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 msgr2=0x7fb758197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb748000c00 tx=0x7fb748004040 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 shutdown_connections 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb7440a2800 0x7fb7440a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb7581033b0 0x7fb75819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7581029f0 0x7fb758198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 --2- 172.21.15.96:0/3700403401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb758101800 0x7fb758197b10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 >> 172.21.15.96:0/3700403401 conn(0x7fb7580fcd90 msgr2=0x7fb758105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 shutdown_connections 2024-03-18T15:38:05.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.418+0000 7fb75f2ac640 1 -- 172.21.15.96:0/3700403401 wait complete. 2024-03-18T15:38:05.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 -- 172.21.15.96:0/3892419650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741018f0 msgr2=0x7f5774101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 --2- 172.21.15.96:0/3892419650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741018f0 0x7f5774101d70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f576802bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 -- 172.21.15.96:0/3892419650 shutdown_connections 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 --2- 172.21.15.96:0/3892419650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57741022b0 0x7f5774108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 --2- 172.21.15.96:0/3892419650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741018f0 0x7f5774101d70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 --2- 172.21.15.96:0/3892419650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5774100700 0x7f5774100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 -- 172.21.15.96:0/3892419650 >> 172.21.15.96:0/3892419650 conn(0x7f57740fbc90 msgr2=0x7f57740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 -- 172.21.15.96:0/3892419650 shutdown_connections 2024-03-18T15:38:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 -- 172.21.15.96:0/3892419650 wait complete. 2024-03-18T15:38:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.534+0000 7f5779509640 1 Processor -- start 2024-03-18T15:38:05.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 -- start start 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774100700 0x7f57741969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57741018f0 0x7f5774196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c380 con 0x7f57741022b0 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c200 con 0x7f5774100700 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5779509640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f577410c500 con 0x7f57741018f0 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55504/0 (socket says 172.21.15.96:55504) 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 -- 172.21.15.96:0/2695235493 learned_addr learned my addr 172.21.15.96:0/2695235493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57727fc640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57741018f0 0x7f5774196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f5772ffd640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774100700 0x7f57741969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 -- 172.21.15.96:0/2695235493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57741018f0 msgr2=0x7f5774196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57741018f0 0x7f5774196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 -- 172.21.15.96:0/2695235493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774100700 msgr2=0x7f57741969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774100700 0x7f57741969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 -- 172.21.15.96:0/2695235493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577419e610 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f57737fe640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f576c00cce0 tx=0x7f576c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.535+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c00f450 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.536+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f576c00fad0 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.536+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577419e900 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.536+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c018c10 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.536+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577419ee20 con 0x7f57741022b0 2024-03-18T15:38:05.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.537+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5774107a40 con 0x7f57741022b0 2024-03-18T15:38:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.537+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f576c00f5f0 con 0x7f57741022b0 2024-03-18T15:38:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.538+0000 7f575bfff640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 0x7f575c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.538+0000 7f5772ffd640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 0x7f575c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.538+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f576c0cef90 con 0x7f57741022b0 2024-03-18T15:38:05.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.538+0000 7f5772ffd640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 0x7f575c0a4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5764005fd0 tx=0x7f5764005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.542+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f576c014030 con 0x7f57741022b0 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/659490690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3700403401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:05.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:05 smithi096 ceph-mon[31930]: pgmap v287: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-03-18T15:38:05.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.760+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f57740623e0 con 0x7f57741022b0 2024-03-18T15:38:05.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.813+0000 7f575bfff640 1 -- 172.21.15.96:0/2695235493 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f576c09c180 con 0x7f57741022b0 2024-03-18T15:38:05.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 msgr2=0x7f575c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 0x7f575c0a4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5764005fd0 tx=0x7f5764005850 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 msgr2=0x7f577419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f576c00cce0 tx=0x7f576c00a620 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 shutdown_connections 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f575c0a2880 0x7f575c0a4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f57741022b0 0x7f577419dfa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57741018f0 0x7f5774196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 --2- 172.21.15.96:0/2695235493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5774100700 0x7f57741969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 >> 172.21.15.96:0/2695235493 conn(0x7f57740fbc90 msgr2=0x7f5774104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 shutdown_connections 2024-03-18T15:38:05.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.816+0000 7f5779509640 1 -- 172.21.15.96:0/2695235493 wait complete. 2024-03-18T15:38:05.829 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-03-18T15:38:05.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- 172.21.15.96:0/1663841854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81022b0 msgr2=0x7f05e8108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 --2- 172.21.15.96:0/1663841854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81022b0 0x7f05e8108b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009ca0 tx=0x7f05e002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- 172.21.15.96:0/1663841854 shutdown_connections 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 --2- 172.21.15.96:0/1663841854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81022b0 0x7f05e8108b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 --2- 172.21.15.96:0/1663841854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e81018f0 0x7f05e8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 --2- 172.21.15.96:0/1663841854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05e8100700 0x7f05e8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- 172.21.15.96:0/1663841854 >> 172.21.15.96:0/1663841854 conn(0x7f05e80fbc70 msgr2=0x7f05e80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- 172.21.15.96:0/1663841854 shutdown_connections 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- 172.21.15.96:0/1663841854 wait complete. 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 Processor -- start 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.934+0000 7f05ed537640 1 -- start start 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e8100700 0x7f05e806a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05e81022b0 0x7f05e819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810c380 con 0x7f05e81018f0 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810c200 con 0x7f05e81022b0 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05e810c500 con 0x7f05e8100700 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55522/0 (socket says 172.21.15.96:55522) 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 -- 172.21.15.96:0/680647387 learned_addr learned my addr 172.21.15.96:0/680647387 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05e6ffd640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e8100700 0x7f05e806a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 -- 172.21.15.96:0/680647387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e8100700 msgr2=0x7f05e806a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e8100700 0x7f05e806a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 -- 172.21.15.96:0/680647387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05e81022b0 msgr2=0x7f05e819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:05.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05e81022b0 0x7f05e819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 -- 172.21.15.96:0/680647387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e819f6c0 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05dffff640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e9f0 tx=0x7f05d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d000cd80 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d0005500 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e819f9b0 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0010690 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.935+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e819ff00 con 0x7f05e81018f0 2024-03-18T15:38:05.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.936+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e8107a20 con 0x7f05e81018f0 2024-03-18T15:38:05.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.937+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f05d00046c0 con 0x7f05e81018f0 2024-03-18T15:38:05.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.938+0000 7f05e4ff9640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 0x7f05d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:05.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.938+0000 7f05e6ffd640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 0x7f05d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:05.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.938+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f05d00cef30 con 0x7f05e81018f0 2024-03-18T15:38:05.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.938+0000 7f05e6ffd640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 0x7f05d40a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05e8101750 tx=0x7f05d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:05.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:05.941+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f05d0053740 con 0x7f05e81018f0 2024-03-18T15:38:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T15:38:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T15:38:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-18T15:38:06.029 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-18T15:38:06.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/659490690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:06.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3700403401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:06.030 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:05 smithi137 ceph-mon[35868]: pgmap v287: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-03-18T15:38:06.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:06.161+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f05e802d340 con 0x7f05e81018f0 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: osdmap e85: 12 total, 12 up, 12 in 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3719850475' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/659490690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3700403401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:06.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:05 smithi193 ceph-mon[36702]: pgmap v287: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 1 op/s 2024-03-18T15:38:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T15:38:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T15:38:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T15:38:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: Cluster is now healthy 2024-03-18T15:38:06.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T15:38:06.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:active 2024-03-18T15:38:06.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:06.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T15:38:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: Cluster is now healthy 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:active 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T15:38:07.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.147+0000 7f05e4ff9640 1 -- 172.21.15.96:0/680647387 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f05d009c120 con 0x7f05e81018f0 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 msgr2=0x7f05d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 0x7f05d40a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05e8101750 tx=0x7f05d8005f90 comp rx=0 tx=0).stop 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 msgr2=0x7f05e819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f05d000e9f0 tx=0x7f05d000edf0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 shutdown_connections 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f05d40a2880 0x7f05d40a4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f05e81022b0 0x7f05e819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05e81018f0 0x7f05e819c7c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 --2- 172.21.15.96:0/680647387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05e8100700 0x7f05e806a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 >> 172.21.15.96:0/680647387 conn(0x7f05e80fbc70 msgr2=0x7f05e8104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:07.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 shutdown_connections 2024-03-18T15:38:07.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.150+0000 7f05ed537640 1 -- 172.21.15.96:0/680647387 wait complete. 2024-03-18T15:38:07.167 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: osdmap e86: 12 total, 12 up, 12 in 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: Cluster is now healthy 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2695235493' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] up:active 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-18T15:38:07.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 -- 172.21.15.96:0/1900756782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 msgr2=0x7fe698108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 --2- 172.21.15.96:0/1900756782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe698108b40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe690009ca0 tx=0x7fe69002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 -- 172.21.15.96:0/1900756782 shutdown_connections 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 --2- 172.21.15.96:0/1900756782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe698108b40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 --2- 172.21.15.96:0/1900756782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 0x7fe698101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 --2- 172.21.15.96:0/1900756782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 0x7fe698100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 -- 172.21.15.96:0/1900756782 >> 172.21.15.96:0/1900756782 conn(0x7fe6980fbc90 msgr2=0x7fe6980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 -- 172.21.15.96:0/1900756782 shutdown_connections 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.275+0000 7fe69de0b640 1 -- 172.21.15.96:0/1900756782 wait complete. 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 Processor -- start 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 -- start start 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 0x7fe69806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:07.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 0x7fe69819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe69810c360 con 0x7fe6981022b0 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe69810c1e0 con 0x7fe6981018f0 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe69de0b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe69810c4e0 con 0x7fe698100700 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55538/0 (socket says 172.21.15.96:55538) 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 -- 172.21.15.96:0/357113409 learned_addr learned my addr 172.21.15.96:0/357113409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe6977fe640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 0x7fe69806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe696ffd640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 0x7fe69819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 -- 172.21.15.96:0/357113409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 msgr2=0x7fe69806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 0x7fe69806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 -- 172.21.15.96:0/357113409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 msgr2=0x7fe69819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 0x7fe69819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:07.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 -- 172.21.15.96:0/357113409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69819f690 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.276+0000 7fe697fff640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe690037040 tx=0x7fe690032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.277+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690041020 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.277+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690034d90 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.277+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69819f8c0 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.277+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690039040 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.277+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69819fdb0 con 0x7fe6981022b0 2024-03-18T15:38:07.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.278+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6981079c0 con 0x7fe6981022b0 2024-03-18T15:38:07.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.279+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe690044020 con 0x7fe6981022b0 2024-03-18T15:38:07.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.279+0000 7fe694ff9640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 0x7fe6800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:07.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.280+0000 7fe6977fe640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 0x7fe6800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:07.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.280+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe6900f0cb0 con 0x7fe6981022b0 2024-03-18T15:38:07.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.280+0000 7fe6977fe640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 0x7fe6800a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe698101750 tx=0x7fe688005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:07.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.283+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe69004b020 con 0x7fe6981022b0 2024-03-18T15:38:07.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:07.497+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe69802d250 con 0x7fe6981022b0 2024-03-18T15:38:07.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T15:38:07.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T15:38:07.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T15:38:07.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-18T15:38:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-18T15:38:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: pgmap v289: 97 pgs: 54 active+clean, 23 creating+peering, 20 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-18T15:38:07.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:07 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T15:38:08.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: pgmap v289: 97 pgs: 54 active+clean, 23 creating+peering, 20 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-18T15:38:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:07 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T15:38:08.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.159+0000 7fe694ff9640 1 -- 172.21.15.96:0/357113409 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fe6900bdea0 con 0x7fe6981022b0 2024-03-18T15:38:08.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 msgr2=0x7fe6800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:08.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 0x7fe6800a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe698101750 tx=0x7fe688005f90 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 msgr2=0x7fe69819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe690037040 tx=0x7fe690032f20 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 shutdown_connections 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6800a2800 0x7fe6800a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022b0 0x7fe69819ed80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6981018f0 0x7fe69819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 --2- 172.21.15.96:0/357113409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe698100700 0x7fe69806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 >> 172.21.15.96:0/357113409 conn(0x7fe6980fbc90 msgr2=0x7fe698104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 shutdown_connections 2024-03-18T15:38:08.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.161+0000 7fe69de0b640 1 -- 172.21.15.96:0/357113409 wait complete. 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/680647387' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: pgmap v289: 97 pgs: 54 active+clean, 23 creating+peering, 20 unknown; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-03-18T15:38:08.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:07 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-18T15:38:08.178 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- 172.21.15.96:0/2891121649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 msgr2=0x7fc48c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 --2- 172.21.15.96:0/2891121649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c108b40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc484009900 tx=0x7fc48402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- 172.21.15.96:0/2891121649 shutdown_connections 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 --2- 172.21.15.96:0/2891121649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c108b40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 --2- 172.21.15.96:0/2891121649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 0x7fc48c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 --2- 172.21.15.96:0/2891121649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- 172.21.15.96:0/2891121649 >> 172.21.15.96:0/2891121649 conn(0x7fc48c0fbc70 msgr2=0x7fc48c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- 172.21.15.96:0/2891121649 shutdown_connections 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- 172.21.15.96:0/2891121649 wait complete. 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 Processor -- start 2024-03-18T15:38:08.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.285+0000 7fc49171b640 1 -- start start 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 0x7fc48c196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c10c380 con 0x7fc48c1022b0 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c10c200 con 0x7fc48c100700 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c10c500 con 0x7fc48c1018f0 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55552/0 (socket says 172.21.15.96:55552) 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 -- 172.21.15.96:0/1736119215 learned_addr learned my addr 172.21.15.96:0/1736119215 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48a7fc640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 0x7fc48c196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 -- 172.21.15.96:0/1736119215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 msgr2=0x7fc48c196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 0x7fc48c196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 -- 172.21.15.96:0/1736119215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 msgr2=0x7fc48c1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 -- 172.21.15.96:0/1736119215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c19e610 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48affd640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc48b7fe640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc484009230 tx=0x7fc484004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484041070 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc484004450 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c19e8a0 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484003dd0 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.286+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c19ede0 con 0x7fc48c1022b0 2024-03-18T15:38:08.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.287+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48c107aa0 con 0x7fc48c1022b0 2024-03-18T15:38:08.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.288+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fc484044070 con 0x7fc48c1022b0 2024-03-18T15:38:08.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.289+0000 7fc473fff640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 0x7fc4740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:08.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.289+0000 7fc48affd640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 0x7fc4740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:08.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.289+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc4840f0380 con 0x7fc48c1022b0 2024-03-18T15:38:08.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.289+0000 7fc48affd640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 0x7fc4740a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc48c101750 tx=0x7fc47c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:08.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.293+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc4840bcf20 con 0x7fc48c1022b0 2024-03-18T15:38:08.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:08.507+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc48c02d3d0 con 0x7fc48c1022b0 2024-03-18T15:38:09.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.165+0000 7fc473fff640 1 -- 172.21.15.96:0/1736119215 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fc4840c0fc0 con 0x7fc48c1022b0 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 msgr2=0x7fc4740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 0x7fc4740a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc48c101750 tx=0x7fc47c009500 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 msgr2=0x7fc48c19dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc484009230 tx=0x7fc484004970 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 shutdown_connections 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc4740a2880 0x7fc4740a4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc48c1022b0 0x7fc48c19dfa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc48c1018f0 0x7fc48c196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 --2- 172.21.15.96:0/1736119215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc48c100700 0x7fc48c1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 >> 172.21.15.96:0/1736119215 conn(0x7fc48c0fbc70 msgr2=0x7fc48c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 shutdown_connections 2024-03-18T15:38:09.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.168+0000 7fc49171b640 1 -- 172.21.15.96:0/1736119215 wait complete. 2024-03-18T15:38:09.184 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-03-18T15:38:09.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-18T15:38:09.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 -- 172.21.15.96:0/1009605546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941029e0 msgr2=0x7f4294102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 --2- 172.21.15.96:0/1009605546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941029e0 0x7f4294102e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4290009900 tx=0x7f429002bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:09.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 -- 172.21.15.96:0/1009605546 shutdown_connections 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 --2- 172.21.15.96:0/1009605546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 0x7f4294109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 --2- 172.21.15.96:0/1009605546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941029e0 0x7f4294102e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 --2- 172.21.15.96:0/1009605546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941017f0 0x7f4294101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 -- 172.21.15.96:0/1009605546 >> 172.21.15.96:0/1009605546 conn(0x7f42940fcd80 msgr2=0x7f42940ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 -- 172.21.15.96:0/1009605546 shutdown_connections 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 -- 172.21.15.96:0/1009605546 wait complete. 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.289+0000 7f429c244640 1 Processor -- start 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- start start 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941029e0 0x7f429419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 0x7f42941a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4294075000 con 0x7f42941017f0 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4294074e80 con 0x7f42941029e0 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4294075180 con 0x7f42941033a0 2024-03-18T15:38:09.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55576/0 (socket says 172.21.15.96:55576) 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 -- 172.21.15.96:0/637261330 learned_addr learned my addr 172.21.15.96:0/637261330 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429a7e1640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 0x7f42941a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f42997df640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941029e0 0x7f429419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 -- 172.21.15.96:0/637261330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 msgr2=0x7f42941a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 0x7f42941a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 -- 172.21.15.96:0/637261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941029e0 msgr2=0x7f429419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941029e0 0x7f429419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 -- 172.21.15.96:0/637261330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941a1960 con 0x7f42941017f0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f4299fe0640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f428400d950 tx=0x7f428400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4284017020 con 0x7f42941017f0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4284009c30 con 0x7f42941017f0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428400f740 con 0x7f42941017f0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941a1c50 con 0x7f42941017f0 2024-03-18T15:38:09.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.290+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a21a0 con 0x7f42941017f0 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.292+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4294108a40 con 0x7f42941017f0 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.293+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7f428401a020 con 0x7f42941017f0 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.293+0000 7f428affd640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 0x7f42800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.293+0000 7f42997df640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 0x7f42800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.294+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f42840ce340 con 0x7f42941017f0 2024-03-18T15:38:09.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.294+0000 7f42997df640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 0x7f42800a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f429002c7b0 tx=0x7f4290005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.298+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4284021070 con 0x7f42941017f0 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:active 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-18T15:38:09.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:active 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-18T15:38:09.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T15:38:09.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.498+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4294061c20 con 0x7f42800a2800 2024-03-18T15:38:09.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.516+0000 7f428affd640 1 -- 172.21.15.96:0/637261330 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4294061c20 con 0x7f42800a2800 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 msgr2=0x7f42800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 0x7f42800a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f429002c7b0 tx=0x7f4290005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 msgr2=0x7f4294199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f428400d950 tx=0x7f428400de10 comp rx=0 tx=0).stop 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 shutdown_connections 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f42800a2800 0x7f42800a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42941033a0 0x7f42941a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42941029e0 0x7f429419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 --2- 172.21.15.96:0/637261330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42941017f0 0x7f4294199d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 >> 172.21.15.96:0/637261330 conn(0x7f42940fcd80 msgr2=0x7f4294105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 shutdown_connections 2024-03-18T15:38:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.518+0000 7f429c244640 1 -- 172.21.15.96:0/637261330 wait complete. 2024-03-18T15:38:09.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-18T15:38:09.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/357113409' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-18T15:38:09.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] up:active 2024-03-18T15:38:09.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-18T15:38:09.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-03-18T15:38:09.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-03-18T15:38:09.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-03-18T15:38:09.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:09 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 -- 172.21.15.96:0/318805443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801018f0 msgr2=0x7fe780101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 --2- 172.21.15.96:0/318805443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801018f0 0x7fe780101d70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe774009900 tx=0x7fe77402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 -- 172.21.15.96:0/318805443 shutdown_connections 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 --2- 172.21.15.96:0/318805443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7801022b0 0x7fe780108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 --2- 172.21.15.96:0/318805443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801018f0 0x7fe780101d70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 --2- 172.21.15.96:0/318805443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe780100700 0x7fe780100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 -- 172.21.15.96:0/318805443 >> 172.21.15.96:0/318805443 conn(0x7fe7800fbc90 msgr2=0x7fe7800fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 -- 172.21.15.96:0/318805443 shutdown_connections 2024-03-18T15:38:09.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.634+0000 7fe785227640 1 -- 172.21.15.96:0/318805443 wait complete. 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 Processor -- start 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 -- start start 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe780100700 0x7fe7801045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7801018f0 0x7fe780102cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe78010c380 con 0x7fe7801022b0 2024-03-18T15:38:09.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe78010c200 con 0x7fe7801018f0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe785227640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe78010c500 con 0x7fe780100700 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7801018f0 0x7fe780102cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55618/0 (socket says 172.21.15.96:55618) 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 -- 172.21.15.96:0/2022743563 learned_addr learned my addr 172.21.15.96:0/2022743563 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe780100700 0x7fe7801045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 -- 172.21.15.96:0/2022743563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe780100700 msgr2=0x7fe7801045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe780100700 0x7fe7801045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 -- 172.21.15.96:0/2022743563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7801018f0 msgr2=0x7fe780102cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.635+0000 7fe77f59e640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7801018f0 0x7fe780102cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe77f59e640 1 -- 172.21.15.96:0/2022743563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe780103a10 con 0x7fe7801022b0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe77f59e640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe77800eb10 tx=0x7fe77800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778004b20 con 0x7fe7801022b0 2024-03-18T15:38:09.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7801a1c90 con 0x7fe7801022b0 2024-03-18T15:38:09.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe778004cc0 con 0x7fe7801022b0 2024-03-18T15:38:09.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778010760 con 0x7fe7801022b0 2024-03-18T15:38:09.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.636+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7801a21b0 con 0x7fe7801022b0 2024-03-18T15:38:09.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.637+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7801a2440 con 0x7fe7801022b0 2024-03-18T15:38:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.638+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fe778005000 con 0x7fe7801022b0 2024-03-18T15:38:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.638+0000 7fe767fff640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 0x7fe7680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.639+0000 7fe77ed9d640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 0x7fe7680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.639+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe7780cf360 con 0x7fe7801022b0 2024-03-18T15:38:09.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.639+0000 7fe77ed9d640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 0x7fe7680a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe780101750 tx=0x7fe770005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.642+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe77809c550 con 0x7fe7801022b0 2024-03-18T15:38:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.846+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe7800fc9d0 con 0x7fe7680a2800 2024-03-18T15:38:09.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.865+0000 7fe767fff640 1 -- 172.21.15.96:0/2022743563 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7800fc9d0 con 0x7fe7680a2800 2024-03-18T15:38:09.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 msgr2=0x7fe7680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 0x7fe7680a4cc0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe780101750 tx=0x7fe770005f90 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 msgr2=0x7fe780103210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe77800eb10 tx=0x7fe77800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 shutdown_connections 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe7680a2800 0x7fe7680a4cc0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801022b0 0x7fe780103210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7801018f0 0x7fe780102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.867+0000 7fe785227640 1 --2- 172.21.15.96:0/2022743563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe780100700 0x7fe7801045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.868+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 >> 172.21.15.96:0/2022743563 conn(0x7fe7800fbc90 msgr2=0x7fe7800fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.868+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 shutdown_connections 2024-03-18T15:38:09.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.868+0000 7fe785227640 1 -- 172.21.15.96:0/2022743563 wait complete. 2024-03-18T15:38:09.879 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:38:09.503+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:38:09.879 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:38:09.503+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:38:09.879 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:38:09.503+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:38:09.879 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:38:09.503+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:38:09.879 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:38:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:38:09.503+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:38:09.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-18T15:38:09.918 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:38:09.526+0000 7f4361e84700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T15:38:09.918 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:38:09.526+0000 7f4361e84700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-18T15:38:09.918 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:38:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:38:09.526+0000 7f4361e84700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-18T15:38:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- 172.21.15.96:0/3176438466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f8100700 msgr2=0x7fd6f8100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/3176438466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f8100700 0x7fd6f8100b20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009900 tx=0x7fd6e802bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- 172.21.15.96:0/3176438466 shutdown_connections 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/3176438466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81022b0 0x7fd6f8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/3176438466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f81018f0 0x7fd6f8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/3176438466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f8100700 0x7fd6f8100b20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- 172.21.15.96:0/3176438466 >> 172.21.15.96:0/3176438466 conn(0x7fd6f80fbc90 msgr2=0x7fd6f80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- 172.21.15.96:0/3176438466 shutdown_connections 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- 172.21.15.96:0/3176438466 wait complete. 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 Processor -- start 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.986+0000 7fd6fdab2640 1 -- start start 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f8100700 0x7fd6f8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 0x7fd6f8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810d4a0 con 0x7fd6f81022b0 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810d320 con 0x7fd6f8100700 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810d620 con 0x7fd6f81018f0 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55640/0 (socket says 172.21.15.96:55640) 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2626304984 learned_addr learned my addr 172.21.15.96:0/2626304984 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f6ffd640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 0x7fd6f8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f77fe640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f8100700 0x7fd6f8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2626304984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 msgr2=0x7fd6f8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 0x7fd6f8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2626304984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f8100700 msgr2=0x7fd6f8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f8100700 0x7fd6f8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2626304984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f819f770 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f6ffd640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 0x7fd6f8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f7fff640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000eb10 tx=0x7fd6f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004b20 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f819fa60 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6f0004cc0 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0010760 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.987+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f819ffb0 con 0x7fd6f81022b0 2024-03-18T15:38:09.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.988+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8107a70 con 0x7fd6f81022b0 2024-03-18T15:38:09.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.989+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 137290+0+0 (secure 0 0 0) 0x7fd6f0005020 con 0x7fd6f81022b0 2024-03-18T15:38:09.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.990+0000 7fd6f4ff9640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 0x7fd6e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:09.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.990+0000 7fd6f77fe640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 0x7fd6e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:09.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.990+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd6f00ceff0 con 0x7fd6f81022b0 2024-03-18T15:38:09.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.990+0000 7fd6f77fe640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 0x7fd6e00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:09.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:09.995+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd6f009c1e0 con 0x7fd6f81022b0 2024-03-18T15:38:10.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.187+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd6f0098150 con 0x7fd6f81022b0 2024-03-18T15:38:10.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.210+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd6f8061c20 con 0x7fd6e00a2800 2024-03-18T15:38:10.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.213+0000 7fd6f4ff9640 1 -- 172.21.15.96:0/2626304984 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd6f8061c20 con 0x7fd6e00a2800 2024-03-18T15:38:10.214 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_0/383f78a8-a96f-4401-88bd-968faea06422 2024-03-18T15:38:10.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 msgr2=0x7fd6e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 0x7fd6e00a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8000c00 tx=0x7fd6e8005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 msgr2=0x7fd6f819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000eb10 tx=0x7fd6f000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 shutdown_connections 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6e00a2800 0x7fd6e00a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6f81022b0 0x7fd6f819f100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6f81018f0 0x7fd6f8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 --2- 172.21.15.96:0/2626304984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd6f8100700 0x7fd6f8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.215+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 >> 172.21.15.96:0/2626304984 conn(0x7fd6f80fbc90 msgr2=0x7fd6f8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.216+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 shutdown_connections 2024-03-18T15:38:10.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.216+0000 7fd6fdab2640 1 -- 172.21.15.96:0/2626304984 wait complete. 2024-03-18T15:38:10.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 -- 172.21.15.96:0/4289904330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 msgr2=0x7f781c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 --2- 172.21.15.96:0/4289904330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c101c10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f780c009900 tx=0x7f780c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 -- 172.21.15.96:0/4289904330 shutdown_connections 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 --2- 172.21.15.96:0/4289904330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1033a0 0x7f781c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 --2- 172.21.15.96:0/4289904330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1029e0 0x7f781c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 --2- 172.21.15.96:0/4289904330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c101c10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 -- 172.21.15.96:0/4289904330 >> 172.21.15.96:0/4289904330 conn(0x7f781c0fcd80 msgr2=0x7f781c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 -- 172.21.15.96:0/4289904330 shutdown_connections 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.333+0000 7f7821cf8640 1 -- 172.21.15.96:0/4289904330 wait complete. 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 Processor -- start 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 -- start start 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1029e0 0x7f781c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1033a0 0x7f781c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10d470 con 0x7f781c1017f0 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10d2f0 con 0x7f781c1033a0 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f7821cf8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10d5f0 con 0x7f781c1029e0 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f781b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f781b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55652/0 (socket says 172.21.15.96:55652) 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f781b7fe640 1 -- 172.21.15.96:0/833769723 learned_addr learned my addr 172.21.15.96:0/833769723 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:10.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f781bfff640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1033a0 0x7f781c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.334+0000 7f781affd640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1029e0 0x7f781c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 -- 172.21.15.96:0/833769723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1029e0 msgr2=0x7f781c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1029e0 0x7f781c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 -- 172.21.15.96:0/833769723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1033a0 msgr2=0x7f781c074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1033a0 0x7f781c074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 -- 172.21.15.96:0/833769723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c0749a0 con 0x7f781c1017f0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f781b7fe640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f780c000c00 tx=0x7f780c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c041070 con 0x7f781c1017f0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780c004450 con 0x7f781c1017f0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1a1c10 con 0x7f781c1017f0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c034440 con 0x7f781c1017f0 2024-03-18T15:38:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.335+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a2160 con 0x7f781c1017f0 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.336+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781c108b30 con 0x7f781c1017f0 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.337+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f780c044070 con 0x7f781c1017f0 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.338+0000 7f7818ff9640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 0x7f78040a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.338+0000 7f781affd640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 0x7f78040a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.338+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f780c0f04a0 con 0x7f781c1017f0 2024-03-18T15:38:10.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.338+0000 7f781affd640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 0x7f78040a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f781c075300 tx=0x7f7810005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:10.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.342+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f780c0bd730 con 0x7f781c1017f0 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:active 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 15 op/s 2024-03-18T15:38:10.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:10 smithi193 ceph-mon[36702]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:active 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 15 op/s 2024-03-18T15:38:10.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:10 smithi096 ceph-mon[31930]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:10.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.543+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f781c061cb0 con 0x7f78040a2890 2024-03-18T15:38:10.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.568+0000 7f7818ff9640 1 -- 172.21.15.96:0/833769723 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f781c061cb0 con 0x7f78040a2890 2024-03-18T15:38:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 msgr2=0x7f78040a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 0x7f78040a4d50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f781c075300 tx=0x7f7810005f90 comp rx=0 tx=0).stop 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 msgr2=0x7f781c075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f780c000c00 tx=0x7f780c02c640 comp rx=0 tx=0).stop 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 shutdown_connections 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78040a2890 0x7f78040a4d50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f781c1033a0 0x7f781c074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f781c1029e0 0x7f781c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 --2- 172.21.15.96:0/833769723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781c1017f0 0x7f781c075590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 >> 172.21.15.96:0/833769723 conn(0x7f781c0fcd80 msgr2=0x7f781c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 shutdown_connections 2024-03-18T15:38:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.570+0000 7f7821cf8640 1 -- 172.21.15.96:0/833769723 wait complete. 2024-03-18T15:38:10.581 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-18T15:38:10.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1736119215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-18T15:38:10.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] up:active 2024-03-18T15:38:10.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-18T15:38:10.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-03-18T15:38:10.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-03-18T15:38:10.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-03-18T15:38:10.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: pgmap v290: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s wr, 15 op/s 2024-03-18T15:38:10.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:10 smithi137 ceph-mon[35868]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 -- 172.21.15.96:0/1754541543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac102290 msgr2=0x7f41ac108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/1754541543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac102290 0x7f41ac108b20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009900 tx=0x7f41a402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 -- 172.21.15.96:0/1754541543 shutdown_connections 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/1754541543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac102290 0x7f41ac108b20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/1754541543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac1018d0 0x7f41ac101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/1754541543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 0x7f41ac100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.693+0000 7f41b0bb9640 1 -- 172.21.15.96:0/1754541543 >> 172.21.15.96:0/1754541543 conn(0x7f41ac0fbc70 msgr2=0x7f41ac0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- 172.21.15.96:0/1754541543 shutdown_connections 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- 172.21.15.96:0/1754541543 wait complete. 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 Processor -- start 2024-03-18T15:38:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- start start 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 0x7f41ac0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac102290 0x7f41ac19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c360 con 0x7f41ac1018d0 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c1e0 con 0x7f41ac102290 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41b0bb9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac10c4e0 con 0x7f41ac1006e0 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.694+0000 7f41a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41aa59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 0x7f41ac0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55684/0 (socket says 172.21.15.96:55684) 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 -- 172.21.15.96:0/2892730319 learned_addr learned my addr 172.21.15.96:0/2892730319 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41aad9d640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac102290 0x7f41ac19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 -- 172.21.15.96:0/2892730319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 msgr2=0x7f41ac0694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 0x7f41ac0694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 -- 172.21.15.96:0/2892730319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac102290 msgr2=0x7f41ac19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac102290 0x7f41ac19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 -- 172.21.15.96:0/2892730319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac19f700 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41a9d9b640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000cd80 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0010910 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac19f9f0 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0018990 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.695+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac19ff40 con 0x7f41ac1018d0 2024-03-18T15:38:10.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.696+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac107a00 con 0x7f41ac1018d0 2024-03-18T15:38:10.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.697+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f41a0010ac0 con 0x7f41ac1018d0 2024-03-18T15:38:10.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.698+0000 7f419b7fe640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 0x7f41940a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:10.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.698+0000 7f41aa59c640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 0x7f41940a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:10.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.698+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f41a00ced90 con 0x7f41ac1018d0 2024-03-18T15:38:10.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.698+0000 7f41aa59c640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 0x7f41940a4d50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41ac101730 tx=0x7f419c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:10.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.702+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f41a009b830 con 0x7f41ac1018d0 2024-03-18T15:38:10.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.905+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f41ac061cb0 con 0x7f41940a2890 2024-03-18T15:38:10.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.909+0000 7f419b7fe640 1 -- 172.21.15.96:0/2892730319 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f41ac061cb0 con 0x7f41940a2890 2024-03-18T15:38:10.910 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_1/a620fed7-ceed-43fc-8a14-043aeb386ec1 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 msgr2=0x7f41940a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 0x7f41940a4d50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41ac101730 tx=0x7f419c009500 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 msgr2=0x7f41ac19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000eeb0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 shutdown_connections 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41940a2890 0x7f41940a4d50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41ac102290 0x7f41ac19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41ac1018d0 0x7f41ac19c800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 --2- 172.21.15.96:0/2892730319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41ac1006e0 0x7f41ac0694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 >> 172.21.15.96:0/2892730319 conn(0x7f41ac0fbc70 msgr2=0x7f41ac104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 shutdown_connections 2024-03-18T15:38:10.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:10.911+0000 7f41b0bb9640 1 -- 172.21.15.96:0/2892730319 wait complete. 2024-03-18T15:38:10.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 -- 172.21.15.96:0/3832218663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 msgr2=0x7f07680ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 --2- 172.21.15.96:0/3832218663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f07680ffd20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0754009900 tx=0x7f075402bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 -- 172.21.15.96:0/3832218663 shutdown_connections 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 --2- 172.21.15.96:0/3832218663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768100260 0x7f0768109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 --2- 172.21.15.96:0/3832218663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f07680ffd20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 --2- 172.21.15.96:0/3832218663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768102e30 0x7f0768103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 -- 172.21.15.96:0/3832218663 >> 172.21.15.96:0/3832218663 conn(0x7f07680fb420 msgr2=0x7f07680fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 -- 172.21.15.96:0/3832218663 shutdown_connections 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.023+0000 7f076ec85640 1 -- 172.21.15.96:0/3832218663 wait complete. 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 Processor -- start 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 -- start start 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768100260 0x7f0768198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 0x7f076819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f076810d500 con 0x7f07680ff8a0 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f076810d380 con 0x7f0768102e30 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f076810d680 con 0x7f0768100260 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55698/0 (socket says 172.21.15.96:55698) 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 -- 172.21.15.96:0/2719306827 learned_addr learned my addr 172.21.15.96:0/2719306827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f0767fff640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768100260 0x7f0768198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076d222640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 0x7f076819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 -- 172.21.15.96:0/2719306827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768100260 msgr2=0x7f0768198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768100260 0x7f0768198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 -- 172.21.15.96:0/2719306827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 msgr2=0x7f076819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 0x7f076819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 -- 172.21.15.96:0/2719306827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076819f730 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076d222640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 0x7f076819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ca21640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f075c00e9c0 tx=0x7f075c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c00cd80 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f075c0035f0 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c010690 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.024+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076819fa20 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.025+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076819fef0 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.026+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0768108cf0 con 0x7f07680ff8a0 2024-03-18T15:38:11.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.026+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f075c0037a0 con 0x7f07680ff8a0 2024-03-18T15:38:11.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.027+0000 7f0765ffb640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 0x7f07580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.027+0000 7f0767fff640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 0x7f07580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.027+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f075c0cee50 con 0x7f07680ff8a0 2024-03-18T15:38:11.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.027+0000 7f0767fff640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 0x7f07580a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f075402c240 tx=0x7f075402df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.031+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f075c09bf40 con 0x7f07680ff8a0 2024-03-18T15:38:11.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.248+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f076802d3f0 con 0x7f07680ff8a0 2024-03-18T15:38:11.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.249+0000 7f0765ffb640 1 -- 172.21.15.96:0/2719306827 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f075c09b840 con 0x7f07680ff8a0 2024-03-18T15:38:11.249 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:11.250 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:38:11.250 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 msgr2=0x7f07580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 0x7f07580a4d50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f075402c240 tx=0x7f075402df90 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 msgr2=0x7f0768197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f075c00e9c0 tx=0x7f075c00ee80 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 shutdown_connections 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f07580a2890 0x7f07580a4d50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0768102e30 0x7f076819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0768100260 0x7f0768198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 --2- 172.21.15.96:0/2719306827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07680ff8a0 0x7f0768197b10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 >> 172.21.15.96:0/2719306827 conn(0x7f07680fb420 msgr2=0x7f0768105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 shutdown_connections 2024-03-18T15:38:11.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.251+0000 7f076ec85640 1 -- 172.21.15.96:0/2719306827 wait complete. 2024-03-18T15:38:11.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:38:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 -- 172.21.15.96:0/40332153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d8100fe0 msgr2=0x7ff9d8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 --2- 172.21.15.96:0/40332153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8101460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800aa10 tx=0x7ff9c802cf70 comp rx=0 tx=0).stop 2024-03-18T15:38:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 -- 172.21.15.96:0/40332153 shutdown_connections 2024-03-18T15:38:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 --2- 172.21.15.96:0/40332153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 0x7ff9d81082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 --2- 172.21.15.96:0/40332153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8101460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 --2- 172.21.15.96:0/40332153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d80ffe90 0x7ff9d81002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 -- 172.21.15.96:0/40332153 >> 172.21.15.96:0/40332153 conn(0x7ff9d80fb3c0 msgr2=0x7ff9d80fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 -- 172.21.15.96:0/40332153 shutdown_connections 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.363+0000 7ff9e0162640 1 -- 172.21.15.96:0/40332153 wait complete. 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 Processor -- start 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 -- start start 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 0x7ff9d8102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810b2c0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810b140 con 0x7ff9d8100fe0 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9e0162640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d810b440 con 0x7ff9d81019a0 2024-03-18T15:38:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55726/0 (socket says 172.21.15.96:55726) 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 -- 172.21.15.96:0/1469647122 learned_addr learned my addr 172.21.15.96:0/1469647122 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9dd6fd640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9de6ff640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 0x7ff9d8102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 -- 172.21.15.96:0/1469647122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 msgr2=0x7ff9d8102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 0x7ff9d8102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 -- 172.21.15.96:0/1469647122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8100fe0 msgr2=0x7ff9d8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.364+0000 7ff9ddefe640 1 -- 172.21.15.96:0/1469647122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d81032f0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9ddefe640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e9f0 tx=0x7ff9d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400cd80 con 0x7ff9d80ffe90 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4010910 con 0x7ff9d80ffe90 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4018990 con 0x7ff9d80ffe90 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d81a1c60 con 0x7ff9d80ffe90 2024-03-18T15:38:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.365+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a21d0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.366+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d81a23a0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.367+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff9d4010ac0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.367+0000 7ff9ceffd640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 0x7ff9c40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.367+0000 7ff9dd6fd640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 0x7ff9c40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.367+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff9d40cecd0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.368+0000 7ff9dd6fd640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 0x7ff9c40a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8000c00 tx=0x7ff9c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.372+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff9d409b770 con 0x7ff9d80ffe90 2024-03-18T15:38:11.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:active 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: Cluster is now healthy 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-18T15:38:11.437 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:active 2024-03-18T15:38:11.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:11 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:38:11.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.550+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9d81a26b0 con 0x7ff9d80ffe90 2024-03-18T15:38:11.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.551+0000 7ff9ceffd640 1 -- 172.21.15.96:0/1469647122 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22171 (secure 0 0 0) 0x7ff9d409f810 con 0x7ff9d80ffe90 2024-03-18T15:38:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:11.553 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":86,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:38:05.670715+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:11.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 msgr2=0x7ff9c40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 0x7ff9c40a4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8000c00 tx=0x7ff9c8005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 msgr2=0x7ff9d8103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e9f0 tx=0x7ff9d400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 shutdown_connections 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c40a2910 0x7ff9c40a4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9d81019a0 0x7ff9d8102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9d8100fe0 0x7ff9d8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 --2- 172.21.15.96:0/1469647122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9d80ffe90 0x7ff9d8103ee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.553+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 >> 172.21.15.96:0/1469647122 conn(0x7ff9d80fb3c0 msgr2=0x7ff9d8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.554+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 shutdown_connections 2024-03-18T15:38:11.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.554+0000 7ff9e0162640 1 -- 172.21.15.96:0/1469647122 wait complete. 2024-03-18T15:38:11.566 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T15:38:11.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:38:11.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:active 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: Cluster is now healthy 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:active 2024-03-18T15:38:11.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:11 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- 172.21.15.96:0/1301613604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f0100700 msgr2=0x7f67f0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 --2- 172.21.15.96:0/1301613604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f0100700 0x7f67f0100b20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009900 tx=0x7f67e002bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- 172.21.15.96:0/1301613604 shutdown_connections 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 --2- 172.21.15.96:0/1301613604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f01022b0 0x7f67f0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 --2- 172.21.15.96:0/1301613604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 0x7f67f0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 --2- 172.21.15.96:0/1301613604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f0100700 0x7f67f0100b20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- 172.21.15.96:0/1301613604 >> 172.21.15.96:0/1301613604 conn(0x7f67f00fbc90 msgr2=0x7f67f00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- 172.21.15.96:0/1301613604 shutdown_connections 2024-03-18T15:38:11.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- 172.21.15.96:0/1301613604 wait complete. 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 Processor -- start 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.667+0000 7f67f79f3640 1 -- start start 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f0100700 0x7f67f0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 0x7f67f0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010d4a0 con 0x7f67f01022b0 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010d320 con 0x7f67f01018f0 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67f010d620 con 0x7f67f0100700 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55744/0 (socket says 172.21.15.96:55744) 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 -- 172.21.15.96:0/2293094146 learned_addr learned my addr 172.21.15.96:0/2293094146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f578f640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f0100700 0x7f67f0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f4f8e640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 0x7f67f0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 -- 172.21.15.96:0/2293094146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f0100700 msgr2=0x7f67f0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f0100700 0x7f67f0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 -- 172.21.15.96:0/2293094146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 msgr2=0x7f67f0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 0x7f67f0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 -- 172.21.15.96:0/2293094146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f019f740 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f5f90640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67e800eb10 tx=0x7f67e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8004b20 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67e8004cc0 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f019fa30 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e8010760 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.668+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f019ff80 con 0x7f67f01022b0 2024-03-18T15:38:11.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.669+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f0107aa0 con 0x7f67f01022b0 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] up:active 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: Cluster is now healthy 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:11.673 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] up:active 2024-03-18T15:38:11.674 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:11 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:38:11.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.670+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f67e8005020 con 0x7f67f01022b0 2024-03-18T15:38:11.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.671+0000 7f67e67fc640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 0x7f67dc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:11.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.671+0000 7f67f578f640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 0x7f67dc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:11.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.671+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f67e80cf120 con 0x7f67f01022b0 2024-03-18T15:38:11.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.671+0000 7f67f578f640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 0x7f67dc0a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000c00 tx=0x7f67e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:11.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.676+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f67e809c210 con 0x7f67f01022b0 2024-03-18T15:38:11.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.849+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67f0061c20 con 0x7f67dc0a2890 2024-03-18T15:38:11.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.857+0000 7f67e67fc640 1 -- 172.21.15.96:0/2293094146 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7f67f0061c20 con 0x7f67dc0a2890 2024-03-18T15:38:11.861 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:11.862 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 msgr2=0x7f67dc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 0x7f67dc0a4d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67e0000c00 tx=0x7f67e0005fd0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 msgr2=0x7f67f019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f67e800eb10 tx=0x7f67e800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 shutdown_connections 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67dc0a2890 0x7f67dc0a4d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67f01022b0 0x7f67f019f0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f01018f0 0x7f67f0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 --2- 172.21.15.96:0/2293094146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67f0100700 0x7f67f0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:11.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 >> 172.21.15.96:0/2293094146 conn(0x7f67f00fbc90 msgr2=0x7f67f0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:11.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 shutdown_connections 2024-03-18T15:38:11.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:11.860+0000 7f67f79f3640 1 -- 172.21.15.96:0/2293094146 wait complete. 2024-03-18T15:38:11.871 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-18T15:38:11.442933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614990,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":326,"num_write_kb":2999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":264,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364572,"kb_used_data":11224,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577252,"statfs":{"total":1151940427776,"available":1151567106048,"internally_reserved":0,"allocated":11493376,"data_stored":7123083,"data_compressed":106791,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24622,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.631354"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051847+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:38:06.051847+0000","last_peered":"2024-03-18T15:38:06.051847+0000","last_clean":"2024-03-18T15:38:06.051847+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:38:06.051847+0000","last_undegraded":"2024-03-18T15:38:06.051847+0000","last_fullsized":"2024-03-18T15:38:06.051847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051277+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:38:06.051277+0000","last_peered":"2024-03-18T15:38:06.051277+0000","last_clean":"2024-03-18T15:38:06.051277+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:38:06.051277+0000","last_undegraded":"2024-03-18T15:38:06.051277+0000","last_fullsized":"2024-03-18T15:38:06.051277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051068+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:38:06.051068+0000","last_peered":"2024-03-18T15:38:06.051068+0000","last_clean":"2024-03-18T15:38:06.051068+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:38:06.051068+0000","last_undegraded":"2024-03-18T15:38:06.051068+0000","last_fullsized":"2024-03-18T15:38:06.051068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052128+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:38:06.052128+0000","last_peered":"2024-03-18T15:38:06.052128+0000","last_clean":"2024-03-18T15:38:06.052128+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:38:06.052128+0000","last_undegraded":"2024-03-18T15:38:06.052128+0000","last_fullsized":"2024-03-18T15:38:06.052128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824360+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:38:05.824360+0000","last_peered":"2024-03-18T15:38:05.824360+0000","last_clean":"2024-03-18T15:38:05.824360+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:38:05.824360+0000","last_undegraded":"2024-03-18T15:38:05.824360+0000","last_fullsized":"2024-03-18T15:38:05.824360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052617+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:38:06.052617+0000","last_peered":"2024-03-18T15:38:06.052617+0000","last_clean":"2024-03-18T15:38:06.052617+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:38:06.052617+0000","last_undegraded":"2024-03-18T15:38:06.052617+0000","last_fullsized":"2024-03-18T15:38:06.052617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051797+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:38:06.051797+0000","last_peered":"2024-03-18T15:38:06.051797+0000","last_clean":"2024-03-18T15:38:06.051797+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:38:06.051797+0000","last_undegraded":"2024-03-18T15:38:06.051797+0000","last_fullsized":"2024-03-18T15:38:06.051797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679218+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:38:05.679218+0000","last_peered":"2024-03-18T15:38:05.679218+0000","last_clean":"2024-03-18T15:38:05.679218+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:38:05.679218+0000","last_undegraded":"2024-03-18T15:38:05.679218+0000","last_fullsized":"2024-03-18T15:38:05.679218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.680005+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:38:05.680005+0000","last_peered":"2024-03-18T15:38:05.680005+0000","last_clean":"2024-03-18T15:38:05.680005+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:38:05.680005+0000","last_undegraded":"2024-03-18T15:38:05.680005+0000","last_fullsized":"2024-03-18T15:38:05.680005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052998+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:38:06.052998+0000","last_peered":"2024-03-18T15:38:06.052998+0000","last_clean":"2024-03-18T15:38:06.052998+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:38:06.052998+0000","last_undegraded":"2024-03-18T15:38:06.052998+0000","last_fullsized":"2024-03-18T15:38:06.052998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052007+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:38:06.052007+0000","last_peered":"2024-03-18T15:38:06.052007+0000","last_clean":"2024-03-18T15:38:06.052007+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:38:06.052007+0000","last_undegraded":"2024-03-18T15:38:06.052007+0000","last_fullsized":"2024-03-18T15:38:06.052007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051936+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:38:06.051936+0000","last_peered":"2024-03-18T15:38:06.051936+0000","last_clean":"2024-03-18T15:38:06.051936+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:38:06.051936+0000","last_undegraded":"2024-03-18T15:38:06.051936+0000","last_fullsized":"2024-03-18T15:38:06.051936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051686+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:38:06.051686+0000","last_peered":"2024-03-18T15:38:06.051686+0000","last_clean":"2024-03-18T15:38:06.051686+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:38:06.051686+0000","last_undegraded":"2024-03-18T15:38:06.051686+0000","last_fullsized":"2024-03-18T15:38:06.051686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051342+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:38:06.051342+0000","last_peered":"2024-03-18T15:38:06.051342+0000","last_clean":"2024-03-18T15:38:06.051342+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:38:06.051342+0000","last_undegraded":"2024-03-18T15:38:06.051342+0000","last_fullsized":"2024-03-18T15:38:06.051342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051246+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:38:06.051246+0000","last_peered":"2024-03-18T15:38:06.051246+0000","last_clean":"2024-03-18T15:38:06.051246+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:38:06.051246+0000","last_undegraded":"2024-03-18T15:38:06.051246+0000","last_fullsized":"2024-03-18T15:38:06.051246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679398+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:38:05.679398+0000","last_peered":"2024-03-18T15:38:05.679398+0000","last_clean":"2024-03-18T15:38:05.679398+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:38:05.679398+0000","last_undegraded":"2024-03-18T15:38:05.679398+0000","last_fullsized":"2024-03-18T15:38:05.679398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051828+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:38:06.051828+0000","last_peered":"2024-03-18T15:38:06.051828+0000","last_clean":"2024-03-18T15:38:06.051828+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:38:06.051828+0000","last_undegraded":"2024-03-18T15:38:06.051828+0000","last_fullsized":"2024-03-18T15:38:06.051828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819672+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:38:05.819672+0000","last_peered":"2024-03-18T15:38:05.819672+0000","last_clean":"2024-03-18T15:38:05.819672+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:38:05.819672+0000","last_undegraded":"2024-03-18T15:38:05.819672+0000","last_fullsized":"2024-03-18T15:38:05.819672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051577+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:38:06.051577+0000","last_peered":"2024-03-18T15:38:06.051577+0000","last_clean":"2024-03-18T15:38:06.051577+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:38:06.051577+0000","last_undegraded":"2024-03-18T15:38:06.051577+0000","last_fullsized":"2024-03-18T15:38:06.051577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.180287+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:38:09.180287+0000","last_peered":"2024-03-18T15:38:09.180287+0000","last_clean":"2024-03-18T15:38:09.180287+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:38:09.180287+0000","last_undegraded":"2024-03-18T15:38:09.180287+0000","last_fullsized":"2024-03-18T15:38:09.180287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679404+0000","last_change":"2024-03-18T15:38:04.692899+0000","last_active":"2024-03-18T15:38:05.679404+0000","last_peered":"2024-03-18T15:38:05.679404+0000","last_clean":"2024-03-18T15:38:05.679404+0000","last_became_active":"2024-03-18T15:38:04.692604+0000","last_became_peered":"2024-03-18T15:38:04.692604+0000","last_unstale":"2024-03-18T15:38:05.679404+0000","last_undegraded":"2024-03-18T15:38:05.679404+0000","last_fullsized":"2024-03-18T15:38:05.679404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:35.353444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679537+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:38:05.679537+0000","last_peered":"2024-03-18T15:38:05.679537+0000","last_clean":"2024-03-18T15:38:05.679537+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:38:05.679537+0000","last_undegraded":"2024-03-18T15:38:05.679537+0000","last_fullsized":"2024-03-18T15:38:05.679537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051610+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:38:06.051610+0000","last_peered":"2024-03-18T15:38:06.051610+0000","last_clean":"2024-03-18T15:38:06.051610+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:38:06.051610+0000","last_undegraded":"2024-03-18T15:38:06.051610+0000","last_fullsized":"2024-03-18T15:38:06.051610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.173187+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:38:08.173187+0000","last_peered":"2024-03-18T15:38:08.173187+0000","last_clean":"2024-03-18T15:38:08.173187+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:38:08.173187+0000","last_undegraded":"2024-03-18T15:38:08.173187+0000","last_fullsized":"2024-03-18T15:38:08.173187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679511+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:38:05.679511+0000","last_peered":"2024-03-18T15:38:05.679511+0000","last_clean":"2024-03-18T15:38:05.679511+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:38:05.679511+0000","last_undegraded":"2024-03-18T15:38:05.679511+0000","last_fullsized":"2024-03-18T15:38:05.679511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824633+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:38:05.824633+0000","last_peered":"2024-03-18T15:38:05.824633+0000","last_clean":"2024-03-18T15:38:05.824633+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:38:05.824633+0000","last_undegraded":"2024-03-18T15:38:05.824633+0000","last_fullsized":"2024-03-18T15:38:05.824633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679950+0000","last_change":"2024-03-18T15:38:04.703055+0000","last_active":"2024-03-18T15:38:05.679950+0000","last_peered":"2024-03-18T15:38:05.679950+0000","last_clean":"2024-03-18T15:38:05.679950+0000","last_became_active":"2024-03-18T15:38:04.702646+0000","last_became_peered":"2024-03-18T15:38:04.702646+0000","last_unstale":"2024-03-18T15:38:05.679950+0000","last_undegraded":"2024-03-18T15:38:05.679950+0000","last_fullsized":"2024-03-18T15:38:05.679950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:53.447695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165828+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:38:07.165828+0000","last_peered":"2024-03-18T15:38:07.165828+0000","last_clean":"2024-03-18T15:38:07.165828+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:38:07.165828+0000","last_undegraded":"2024-03-18T15:38:07.165828+0000","last_fullsized":"2024-03-18T15:38:07.165828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824410+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:38:05.824410+0000","last_peered":"2024-03-18T15:38:05.824410+0000","last_clean":"2024-03-18T15:38:05.824410+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:38:05.824410+0000","last_undegraded":"2024-03-18T15:38:05.824410+0000","last_fullsized":"2024-03-18T15:38:05.824410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.181058+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:38:09.181058+0000","last_peered":"2024-03-18T15:38:09.181058+0000","last_clean":"2024-03-18T15:38:09.181058+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:38:09.181058+0000","last_undegraded":"2024-03-18T15:38:09.181058+0000","last_fullsized":"2024-03-18T15:38:09.181058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679946+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:38:05.679946+0000","last_peered":"2024-03-18T15:38:05.679946+0000","last_clean":"2024-03-18T15:38:05.679946+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:38:05.679946+0000","last_undegraded":"2024-03-18T15:38:05.679946+0000","last_fullsized":"2024-03-18T15:38:05.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.050898+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:38:06.050898+0000","last_peered":"2024-03-18T15:38:06.050898+0000","last_clean":"2024-03-18T15:38:06.050898+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:38:06.050898+0000","last_undegraded":"2024-03-18T15:38:06.050898+0000","last_fullsized":"2024-03-18T15:38:06.050898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679226+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:38:05.679226+0000","last_peered":"2024-03-18T15:38:05.679226+0000","last_clean":"2024-03-18T15:38:05.679226+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:38:05.679226+0000","last_undegraded":"2024-03-18T15:38:05.679226+0000","last_fullsized":"2024-03-18T15:38:05.679226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679214+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:38:05.679214+0000","last_peered":"2024-03-18T15:38:05.679214+0000","last_clean":"2024-03-18T15:38:05.679214+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:38:05.679214+0000","last_undegraded":"2024-03-18T15:38:05.679214+0000","last_fullsized":"2024-03-18T15:38:05.679214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819481+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:38:05.819481+0000","last_peered":"2024-03-18T15:38:05.819481+0000","last_clean":"2024-03-18T15:38:05.819481+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:38:05.819481+0000","last_undegraded":"2024-03-18T15:38:05.819481+0000","last_fullsized":"2024-03-18T15:38:05.819481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.163292+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:38:07.163292+0000","last_peered":"2024-03-18T15:38:07.163292+0000","last_clean":"2024-03-18T15:38:07.163292+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:38:07.163292+0000","last_undegraded":"2024-03-18T15:38:07.163292+0000","last_fullsized":"2024-03-18T15:38:07.163292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679845+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:38:05.679845+0000","last_peered":"2024-03-18T15:38:05.679845+0000","last_clean":"2024-03-18T15:38:05.679845+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:38:05.679845+0000","last_undegraded":"2024-03-18T15:38:05.679845+0000","last_fullsized":"2024-03-18T15:38:05.679845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.164228+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:38:07.164228+0000","last_peered":"2024-03-18T15:38:07.164228+0000","last_clean":"2024-03-18T15:38:07.164228+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:38:07.164228+0000","last_undegraded":"2024-03-18T15:38:07.164228+0000","last_fullsized":"2024-03-18T15:38:07.164228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051551+0000","last_change":"2024-03-18T15:38:04.697178+0000","last_active":"2024-03-18T15:38:06.051551+0000","last_peered":"2024-03-18T15:38:06.051551+0000","last_clean":"2024-03-18T15:38:06.051551+0000","last_became_active":"2024-03-18T15:38:04.696918+0000","last_became_peered":"2024-03-18T15:38:04.696918+0000","last_unstale":"2024-03-18T15:38:06.051551+0000","last_undegraded":"2024-03-18T15:38:06.051551+0000","last_fullsized":"2024-03-18T15:38:06.051551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:45:37.731149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679840+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:38:05.679840+0000","last_peered":"2024-03-18T15:38:05.679840+0000","last_clean":"2024-03-18T15:38:05.679840+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:38:05.679840+0000","last_undegraded":"2024-03-18T15:38:05.679840+0000","last_fullsized":"2024-03-18T15:38:05.679840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679759+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:38:05.679759+0000","last_peered":"2024-03-18T15:38:05.679759+0000","last_clean":"2024-03-18T15:38:05.679759+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:38:05.679759+0000","last_undegraded":"2024-03-18T15:38:05.679759+0000","last_fullsized":"2024-03-18T15:38:05.679759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:10.200004+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:38:10.200004+0000","last_peered":"2024-03-18T15:38:10.200004+0000","last_clean":"2024-03-18T15:38:10.200004+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:38:10.200004+0000","last_undegraded":"2024-03-18T15:38:10.200004+0000","last_fullsized":"2024-03-18T15:38:10.200004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824648+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:38:05.824648+0000","last_peered":"2024-03-18T15:38:05.824648+0000","last_clean":"2024-03-18T15:38:05.824648+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:38:05.824648+0000","last_undegraded":"2024-03-18T15:38:05.824648+0000","last_fullsized":"2024-03-18T15:38:05.824648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.175151+0000","last_change":"2024-03-18T15:38:03.834819+0000","last_active":"2024-03-18T15:38:08.175151+0000","last_peered":"2024-03-18T15:38:08.175151+0000","last_clean":"2024-03-18T15:38:08.175151+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:38:08.175151+0000","last_undegraded":"2024-03-18T15:38:08.175151+0000","last_fullsized":"2024-03-18T15:38:08.175151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:02:11.448328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051269+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:38:06.051269+0000","last_peered":"2024-03-18T15:38:06.051269+0000","last_clean":"2024-03-18T15:38:06.051269+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:38:06.051269+0000","last_undegraded":"2024-03-18T15:38:06.051269+0000","last_fullsized":"2024-03-18T15:38:06.051269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165598+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:38:07.165598+0000","last_peered":"2024-03-18T15:38:07.165598+0000","last_clean":"2024-03-18T15:38:07.165598+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:38:07.165598+0000","last_undegraded":"2024-03-18T15:38:07.165598+0000","last_fullsized":"2024-03-18T15:38:07.165598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052145+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:38:06.052145+0000","last_peered":"2024-03-18T15:38:06.052145+0000","last_clean":"2024-03-18T15:38:06.052145+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:38:06.052145+0000","last_undegraded":"2024-03-18T15:38:06.052145+0000","last_fullsized":"2024-03-18T15:38:06.052145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.174803+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:38:08.174803+0000","last_peered":"2024-03-18T15:38:08.174803+0000","last_clean":"2024-03-18T15:38:08.174803+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:38:08.174803+0000","last_undegraded":"2024-03-18T15:38:08.174803+0000","last_fullsized":"2024-03-18T15:38:08.174803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051335+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:38:06.051335+0000","last_peered":"2024-03-18T15:38:06.051335+0000","last_clean":"2024-03-18T15:38:06.051335+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:38:06.051335+0000","last_undegraded":"2024-03-18T15:38:06.051335+0000","last_fullsized":"2024-03-18T15:38:06.051335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.176484+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:38:08.176484+0000","last_peered":"2024-03-18T15:38:08.176484+0000","last_clean":"2024-03-18T15:38:08.176484+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:38:08.176484+0000","last_undegraded":"2024-03-18T15:38:08.176484+0000","last_fullsized":"2024-03-18T15:38:08.176484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679551+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:38:05.679551+0000","last_peered":"2024-03-18T15:38:05.679551+0000","last_clean":"2024-03-18T15:38:05.679551+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:38:05.679551+0000","last_undegraded":"2024-03-18T15:38:05.679551+0000","last_fullsized":"2024-03-18T15:38:05.679551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824640+0000","last_change":"2024-03-18T15:38:03.831562+0000","last_active":"2024-03-18T15:38:05.824640+0000","last_peered":"2024-03-18T15:38:05.824640+0000","last_clean":"2024-03-18T15:38:05.824640+0000","last_became_active":"2024-03-18T15:38:03.831204+0000","last_became_peered":"2024-03-18T15:38:03.831204+0000","last_unstale":"2024-03-18T15:38:05.824640+0000","last_undegraded":"2024-03-18T15:38:05.824640+0000","last_fullsized":"2024-03-18T15:38:05.824640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:22:14.867492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051484+0000","last_change":"2024-03-18T15:38:04.693705+0000","last_active":"2024-03-18T15:38:06.051484+0000","last_peered":"2024-03-18T15:38:06.051484+0000","last_clean":"2024-03-18T15:38:06.051484+0000","last_became_active":"2024-03-18T15:38:04.693283+0000","last_became_peered":"2024-03-18T15:38:04.693283+0000","last_unstale":"2024-03-18T15:38:06.051484+0000","last_undegraded":"2024-03-18T15:38:06.051484+0000","last_fullsized":"2024-03-18T15:38:06.051484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:52:56.238305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819572+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:38:05.819572+0000","last_peered":"2024-03-18T15:38:05.819572+0000","last_clean":"2024-03-18T15:38:05.819572+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:38:05.819572+0000","last_undegraded":"2024-03-18T15:38:05.819572+0000","last_fullsized":"2024-03-18T15:38:05.819572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052937+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:38:06.052937+0000","last_peered":"2024-03-18T15:38:06.052937+0000","last_clean":"2024-03-18T15:38:06.052937+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:38:06.052937+0000","last_undegraded":"2024-03-18T15:38:06.052937+0000","last_fullsized":"2024-03-18T15:38:06.052937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051428+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:38:06.051428+0000","last_peered":"2024-03-18T15:38:06.051428+0000","last_clean":"2024-03-18T15:38:06.051428+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:38:06.051428+0000","last_undegraded":"2024-03-18T15:38:06.051428+0000","last_fullsized":"2024-03-18T15:38:06.051428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052788+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:38:06.052788+0000","last_peered":"2024-03-18T15:38:06.052788+0000","last_clean":"2024-03-18T15:38:06.052788+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:38:06.052788+0000","last_undegraded":"2024-03-18T15:38:06.052788+0000","last_fullsized":"2024-03-18T15:38:06.052788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679489+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:38:05.679489+0000","last_peered":"2024-03-18T15:38:05.679489+0000","last_clean":"2024-03-18T15:38:05.679489+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:38:05.679489+0000","last_undegraded":"2024-03-18T15:38:05.679489+0000","last_fullsized":"2024-03-18T15:38:05.679489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.162720+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:38:07.162720+0000","last_peered":"2024-03-18T15:38:07.162720+0000","last_clean":"2024-03-18T15:38:07.162720+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:38:07.162720+0000","last_undegraded":"2024-03-18T15:38:07.162720+0000","last_fullsized":"2024-03-18T15:38:07.162720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679536+0000","last_change":"2024-03-18T15:38:04.689717+0000","last_active":"2024-03-18T15:38:05.679536+0000","last_peered":"2024-03-18T15:38:05.679536+0000","last_clean":"2024-03-18T15:38:05.679536+0000","last_became_active":"2024-03-18T15:38:04.689352+0000","last_became_peered":"2024-03-18T15:38:04.689352+0000","last_unstale":"2024-03-18T15:38:05.679536+0000","last_undegraded":"2024-03-18T15:38:05.679536+0000","last_fullsized":"2024-03-18T15:38:05.679536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:58.058080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165059+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:38:07.165059+0000","last_peered":"2024-03-18T15:38:07.165059+0000","last_clean":"2024-03-18T15:38:07.165059+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:38:07.165059+0000","last_undegraded":"2024-03-18T15:38:07.165059+0000","last_fullsized":"2024-03-18T15:38:07.165059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051535+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:38:06.051535+0000","last_peered":"2024-03-18T15:38:06.051535+0000","last_clean":"2024-03-18T15:38:06.051535+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:38:06.051535+0000","last_undegraded":"2024-03-18T15:38:06.051535+0000","last_fullsized":"2024-03-18T15:38:06.051535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.050851+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:38:06.050851+0000","last_peered":"2024-03-18T15:38:06.050851+0000","last_clean":"2024-03-18T15:38:06.050851+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:38:06.050851+0000","last_undegraded":"2024-03-18T15:38:06.050851+0000","last_fullsized":"2024-03-18T15:38:06.050851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.173915+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:38:08.173915+0000","last_peered":"2024-03-18T15:38:08.173915+0000","last_clean":"2024-03-18T15:38:08.173915+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:38:08.173915+0000","last_undegraded":"2024-03-18T15:38:08.173915+0000","last_fullsized":"2024-03-18T15:38:08.173915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824744+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:38:05.824744+0000","last_peered":"2024-03-18T15:38:05.824744+0000","last_clean":"2024-03-18T15:38:05.824744+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:38:05.824744+0000","last_undegraded":"2024-03-18T15:38:05.824744+0000","last_fullsized":"2024-03-18T15:38:05.824744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051637+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:38:06.051637+0000","last_peered":"2024-03-18T15:38:06.051637+0000","last_clean":"2024-03-18T15:38:06.051637+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:38:06.051637+0000","last_undegraded":"2024-03-18T15:38:06.051637+0000","last_fullsized":"2024-03-18T15:38:06.051637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.180964+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:38:09.180964+0000","last_peered":"2024-03-18T15:38:09.180964+0000","last_clean":"2024-03-18T15:38:09.180964+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:38:09.180964+0000","last_undegraded":"2024-03-18T15:38:09.180964+0000","last_fullsized":"2024-03-18T15:38:09.180964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679531+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:38:05.679531+0000","last_peered":"2024-03-18T15:38:05.679531+0000","last_clean":"2024-03-18T15:38:05.679531+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:38:05.679531+0000","last_undegraded":"2024-03-18T15:38:05.679531+0000","last_fullsized":"2024-03-18T15:38:05.679531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.163439+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:38:07.163439+0000","last_peered":"2024-03-18T15:38:07.163439+0000","last_clean":"2024-03-18T15:38:07.163439+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:38:07.163439+0000","last_undegraded":"2024-03-18T15:38:07.163439+0000","last_fullsized":"2024-03-18T15:38:07.163439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051189+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:38:06.051189+0000","last_peered":"2024-03-18T15:38:06.051189+0000","last_clean":"2024-03-18T15:38:06.051189+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:38:06.051189+0000","last_undegraded":"2024-03-18T15:38:06.051189+0000","last_fullsized":"2024-03-18T15:38:06.051189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.162336+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:38:07.162336+0000","last_peered":"2024-03-18T15:38:07.162336+0000","last_clean":"2024-03-18T15:38:07.162336+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:38:07.162336+0000","last_undegraded":"2024-03-18T15:38:07.162336+0000","last_fullsized":"2024-03-18T15:38:07.162336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824372+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:38:05.824372+0000","last_peered":"2024-03-18T15:38:05.824372+0000","last_clean":"2024-03-18T15:38:05.824372+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:38:05.824372+0000","last_undegraded":"2024-03-18T15:38:05.824372+0000","last_fullsized":"2024-03-18T15:38:05.824372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165304+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:38:07.165304+0000","last_peered":"2024-03-18T15:38:07.165304+0000","last_clean":"2024-03-18T15:38:07.165304+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:38:07.165304+0000","last_undegraded":"2024-03-18T15:38:07.165304+0000","last_fullsized":"2024-03-18T15:38:07.165304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051077+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:38:06.051077+0000","last_peered":"2024-03-18T15:38:06.051077+0000","last_clean":"2024-03-18T15:38:06.051077+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:38:06.051077+0000","last_undegraded":"2024-03-18T15:38:06.051077+0000","last_fullsized":"2024-03-18T15:38:06.051077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051329+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:38:06.051329+0000","last_peered":"2024-03-18T15:38:06.051329+0000","last_clean":"2024-03-18T15:38:06.051329+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:38:06.051329+0000","last_undegraded":"2024-03-18T15:38:06.051329+0000","last_fullsized":"2024-03-18T15:38:06.051329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679374+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:38:05.679374+0000","last_peered":"2024-03-18T15:38:05.679374+0000","last_clean":"2024-03-18T15:38:05.679374+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:38:05.679374+0000","last_undegraded":"2024-03-18T15:38:05.679374+0000","last_fullsized":"2024-03-18T15:38:05.679374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165484+0000","last_change":"2024-03-18T15:38:03.833151+0000","last_active":"2024-03-18T15:38:07.165484+0000","last_peered":"2024-03-18T15:38:07.165484+0000","last_clean":"2024-03-18T15:38:07.165484+0000","last_became_active":"2024-03-18T15:38:03.832775+0000","last_became_peered":"2024-03-18T15:38:03.832775+0000","last_unstale":"2024-03-18T15:38:07.165484+0000","last_undegraded":"2024-03-18T15:38:07.165484+0000","last_fullsized":"2024-03-18T15:38:07.165484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:13.936881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051550+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:38:06.051550+0000","last_peered":"2024-03-18T15:38:06.051550+0000","last_clean":"2024-03-18T15:38:06.051550+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:38:06.051550+0000","last_undegraded":"2024-03-18T15:38:06.051550+0000","last_fullsized":"2024-03-18T15:38:06.051550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.174761+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:38:08.174761+0000","last_peered":"2024-03-18T15:38:08.174761+0000","last_clean":"2024-03-18T15:38:08.174761+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:38:08.174761+0000","last_undegraded":"2024-03-18T15:38:08.174761+0000","last_fullsized":"2024-03-18T15:38:08.174761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052115+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:38:06.052115+0000","last_peered":"2024-03-18T15:38:06.052115+0000","last_clean":"2024-03-18T15:38:06.052115+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:38:06.052115+0000","last_undegraded":"2024-03-18T15:38:06.052115+0000","last_fullsized":"2024-03-18T15:38:06.052115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679556+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:38:05.679556+0000","last_peered":"2024-03-18T15:38:05.679556+0000","last_clean":"2024-03-18T15:38:05.679556+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:38:05.679556+0000","last_undegraded":"2024-03-18T15:38:05.679556+0000","last_fullsized":"2024-03-18T15:38:05.679556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051399+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:38:06.051399+0000","last_peered":"2024-03-18T15:38:06.051399+0000","last_clean":"2024-03-18T15:38:06.051399+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:38:06.051399+0000","last_undegraded":"2024-03-18T15:38:06.051399+0000","last_fullsized":"2024-03-18T15:38:06.051399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051687+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:38:06.051687+0000","last_peered":"2024-03-18T15:38:06.051687+0000","last_clean":"2024-03-18T15:38:06.051687+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:38:06.051687+0000","last_undegraded":"2024-03-18T15:38:06.051687+0000","last_fullsized":"2024-03-18T15:38:06.051687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051815+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:38:06.051815+0000","last_peered":"2024-03-18T15:38:06.051815+0000","last_clean":"2024-03-18T15:38:06.051815+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:38:06.051815+0000","last_undegraded":"2024-03-18T15:38:06.051815+0000","last_fullsized":"2024-03-18T15:38:06.051815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051102+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:38:06.051102+0000","last_peered":"2024-03-18T15:38:06.051102+0000","last_clean":"2024-03-18T15:38:06.051102+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:38:06.051102+0000","last_undegraded":"2024-03-18T15:38:06.051102+0000","last_fullsized":"2024-03-18T15:38:06.051102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051246+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:38:06.051246+0000","last_peered":"2024-03-18T15:38:06.051246+0000","last_clean":"2024-03-18T15:38:06.051246+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:38:06.051246+0000","last_undegraded":"2024-03-18T15:38:06.051246+0000","last_fullsized":"2024-03-18T15:38:06.051246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051279+0000","last_change":"2024-03-18T15:38:04.684780+0000","last_active":"2024-03-18T15:38:06.051279+0000","last_peered":"2024-03-18T15:38:06.051279+0000","last_clean":"2024-03-18T15:38:06.051279+0000","last_became_active":"2024-03-18T15:38:04.684376+0000","last_became_peered":"2024-03-18T15:38:04.684376+0000","last_unstale":"2024-03-18T15:38:06.051279+0000","last_undegraded":"2024-03-18T15:38:06.051279+0000","last_fullsized":"2024-03-18T15:38:06.051279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:47:40.082093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051589+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:38:06.051589+0000","last_peered":"2024-03-18T15:38:06.051589+0000","last_clean":"2024-03-18T15:38:06.051589+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:38:06.051589+0000","last_undegraded":"2024-03-18T15:38:06.051589+0000","last_fullsized":"2024-03-18T15:38:06.051589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051485+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:38:06.051485+0000","last_peered":"2024-03-18T15:38:06.051485+0000","last_clean":"2024-03-18T15:38:06.051485+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:38:06.051485+0000","last_undegraded":"2024-03-18T15:38:06.051485+0000","last_fullsized":"2024-03-18T15:38:06.051485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051421+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:38:06.051421+0000","last_peered":"2024-03-18T15:38:06.051421+0000","last_clean":"2024-03-18T15:38:06.051421+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:38:06.051421+0000","last_undegraded":"2024-03-18T15:38:06.051421+0000","last_fullsized":"2024-03-18T15:38:06.051421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051708+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:38:06.051708+0000","last_peered":"2024-03-18T15:38:06.051708+0000","last_clean":"2024-03-18T15:38:06.051708+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:38:06.051708+0000","last_undegraded":"2024-03-18T15:38:06.051708+0000","last_fullsized":"2024-03-18T15:38:06.051708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679510+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:38:05.679510+0000","last_peered":"2024-03-18T15:38:05.679510+0000","last_clean":"2024-03-18T15:38:05.679510+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:38:05.679510+0000","last_undegraded":"2024-03-18T15:38:05.679510+0000","last_fullsized":"2024-03-18T15:38:05.679510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679580+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:38:05.679580+0000","last_peered":"2024-03-18T15:38:05.679580+0000","last_clean":"2024-03-18T15:38:05.679580+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:38:05.679580+0000","last_undegraded":"2024-03-18T15:38:05.679580+0000","last_fullsized":"2024-03-18T15:38:05.679580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679068+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:38:05.679068+0000","last_peered":"2024-03-18T15:38:05.679068+0000","last_clean":"2024-03-18T15:38:05.679068+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:38:05.679068+0000","last_undegraded":"2024-03-18T15:38:05.679068+0000","last_fullsized":"2024-03-18T15:38:05.679068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679641+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:38:05.679641+0000","last_peered":"2024-03-18T15:38:05.679641+0000","last_clean":"2024-03-18T15:38:05.679641+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:38:05.679641+0000","last_undegraded":"2024-03-18T15:38:05.679641+0000","last_fullsized":"2024-03-18T15:38:05.679641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051726+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:38:06.051726+0000","last_peered":"2024-03-18T15:38:06.051726+0000","last_clean":"2024-03-18T15:38:06.051726+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:38:06.051726+0000","last_undegraded":"2024-03-18T15:38:06.051726+0000","last_fullsized":"2024-03-18T15:38:06.051726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051267+0000","last_change":"2024-03-18T15:38:04.699503+0000","last_active":"2024-03-18T15:38:06.051267+0000","last_peered":"2024-03-18T15:38:06.051267+0000","last_clean":"2024-03-18T15:38:06.051267+0000","last_became_active":"2024-03-18T15:38:04.699163+0000","last_became_peered":"2024-03-18T15:38:04.699163+0000","last_unstale":"2024-03-18T15:38:06.051267+0000","last_undegraded":"2024-03-18T15:38:06.051267+0000","last_fullsized":"2024-03-18T15:38:06.051267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:18.868944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24505,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":54082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1056131,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1035175,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":464492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":445977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1034859,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:38:11.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:38:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 -- 172.21.15.96:0/3827895366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 msgr2=0x7f4400101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 --2- 172.21.15.96:0/3827895366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400101d70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009900 tx=0x7f43e802bc30 comp rx=0 tx=0).stop 2024-03-18T15:38:12.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 -- 172.21.15.96:0/3827895366 shutdown_connections 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 --2- 172.21.15.96:0/3827895366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44001022b0 0x7f4400108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 --2- 172.21.15.96:0/3827895366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400101d70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 --2- 172.21.15.96:0/3827895366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4400100700 0x7f4400100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 -- 172.21.15.96:0/3827895366 >> 172.21.15.96:0/3827895366 conn(0x7f44000fbc90 msgr2=0x7f44000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 -- 172.21.15.96:0/3827895366 shutdown_connections 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.012+0000 7f44052f8640 1 -- 172.21.15.96:0/3827895366 wait complete. 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 Processor -- start 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 -- start start 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4400100700 0x7f4400197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44001022b0 0x7f440019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f440010c360 con 0x7f44001018f0 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f440010c1e0 con 0x7f4400100700 2024-03-18T15:38:12.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f44052f8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f440010c4e0 con 0x7f44001022b0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55764/0 (socket says 172.21.15.96:55764) 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 -- 172.21.15.96:0/2193063581 learned_addr learned my addr 172.21.15.96:0/2193063581 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43feffd640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4400100700 0x7f4400197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43ff7fe640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44001022b0 0x7f440019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 -- 172.21.15.96:0/2193063581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44001022b0 msgr2=0x7f440019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44001022b0 0x7f440019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 -- 172.21.15.96:0/2193063581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4400100700 msgr2=0x7f4400197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4400100700 0x7f4400197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.013+0000 7f43f7fff640 1 -- 172.21.15.96:0/2193063581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440019f740 con 0x7f44001018f0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f43f7fff640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009230 tx=0x7f43e8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8041070 con 0x7f44001018f0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440019f9d0 con 0x7f44001018f0 2024-03-18T15:38:12.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e80044d0 con 0x7f44001018f0 2024-03-18T15:38:12.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e8034440 con 0x7f44001018f0 2024-03-18T15:38:12.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.014+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440019ff10 con 0x7f44001018f0 2024-03-18T15:38:12.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.015+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44001a00e0 con 0x7f44001018f0 2024-03-18T15:38:12.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.016+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f43e8044070 con 0x7f44001018f0 2024-03-18T15:38:12.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.016+0000 7f43fcff9640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 0x7f43ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:12.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.016+0000 7f43feffd640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 0x7f43ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:12.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.017+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f43e80f0500 con 0x7f44001018f0 2024-03-18T15:38:12.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.017+0000 7f43feffd640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 0x7f43ec0a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005fd0 tx=0x7f43f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:12.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.021+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43e80bd0b0 con 0x7f44001018f0 2024-03-18T15:38:12.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.196+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f440006a540 con 0x7f43ec0a2890 2024-03-18T15:38:12.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.203+0000 7f43fcff9640 1 -- 172.21.15.96:0/2193063581 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258903 (secure 0 0 0) 0x7f440006a540 con 0x7f43ec0a2890 2024-03-18T15:38:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:12.208 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:38:12.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 msgr2=0x7f43ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 0x7f43ec0a4d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005fd0 tx=0x7f43f000b410 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 msgr2=0x7f4400198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009230 tx=0x7f43e8005d50 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 shutdown_connections 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43ec0a2890 0x7f43ec0a4d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44001022b0 0x7f440019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44001018f0 0x7f4400198060 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 --2- 172.21.15.96:0/2193063581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4400100700 0x7f4400197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 >> 172.21.15.96:0/2193063581 conn(0x7f44000fbc90 msgr2=0x7f4400069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 shutdown_connections 2024-03-18T15:38:12.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:12.207+0000 7f44052f8640 1 -- 172.21.15.96:0/2193063581 wait complete. 2024-03-18T15:38:12.218 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-03-18T15:38:11.442933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614990,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":326,"num_write_kb":2999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":264,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364572,"kb_used_data":11224,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577252,"statfs":{"total":1151940427776,"available":1151567106048,"internally_reserved":0,"allocated":11493376,"data_stored":7123083,"data_compressed":106791,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24622,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.631354"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051847+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:38:06.051847+0000","last_peered":"2024-03-18T15:38:06.051847+0000","last_clean":"2024-03-18T15:38:06.051847+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:38:06.051847+0000","last_undegraded":"2024-03-18T15:38:06.051847+0000","last_fullsized":"2024-03-18T15:38:06.051847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051277+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:38:06.051277+0000","last_peered":"2024-03-18T15:38:06.051277+0000","last_clean":"2024-03-18T15:38:06.051277+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:38:06.051277+0000","last_undegraded":"2024-03-18T15:38:06.051277+0000","last_fullsized":"2024-03-18T15:38:06.051277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051068+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:38:06.051068+0000","last_peered":"2024-03-18T15:38:06.051068+0000","last_clean":"2024-03-18T15:38:06.051068+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:38:06.051068+0000","last_undegraded":"2024-03-18T15:38:06.051068+0000","last_fullsized":"2024-03-18T15:38:06.051068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052128+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:38:06.052128+0000","last_peered":"2024-03-18T15:38:06.052128+0000","last_clean":"2024-03-18T15:38:06.052128+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:38:06.052128+0000","last_undegraded":"2024-03-18T15:38:06.052128+0000","last_fullsized":"2024-03-18T15:38:06.052128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824360+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:38:05.824360+0000","last_peered":"2024-03-18T15:38:05.824360+0000","last_clean":"2024-03-18T15:38:05.824360+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:38:05.824360+0000","last_undegraded":"2024-03-18T15:38:05.824360+0000","last_fullsized":"2024-03-18T15:38:05.824360+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052617+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:38:06.052617+0000","last_peered":"2024-03-18T15:38:06.052617+0000","last_clean":"2024-03-18T15:38:06.052617+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:38:06.052617+0000","last_undegraded":"2024-03-18T15:38:06.052617+0000","last_fullsized":"2024-03-18T15:38:06.052617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051797+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:38:06.051797+0000","last_peered":"2024-03-18T15:38:06.051797+0000","last_clean":"2024-03-18T15:38:06.051797+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:38:06.051797+0000","last_undegraded":"2024-03-18T15:38:06.051797+0000","last_fullsized":"2024-03-18T15:38:06.051797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679218+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:38:05.679218+0000","last_peered":"2024-03-18T15:38:05.679218+0000","last_clean":"2024-03-18T15:38:05.679218+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:38:05.679218+0000","last_undegraded":"2024-03-18T15:38:05.679218+0000","last_fullsized":"2024-03-18T15:38:05.679218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.680005+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:38:05.680005+0000","last_peered":"2024-03-18T15:38:05.680005+0000","last_clean":"2024-03-18T15:38:05.680005+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:38:05.680005+0000","last_undegraded":"2024-03-18T15:38:05.680005+0000","last_fullsized":"2024-03-18T15:38:05.680005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052998+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:38:06.052998+0000","last_peered":"2024-03-18T15:38:06.052998+0000","last_clean":"2024-03-18T15:38:06.052998+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:38:06.052998+0000","last_undegraded":"2024-03-18T15:38:06.052998+0000","last_fullsized":"2024-03-18T15:38:06.052998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052007+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:38:06.052007+0000","last_peered":"2024-03-18T15:38:06.052007+0000","last_clean":"2024-03-18T15:38:06.052007+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:38:06.052007+0000","last_undegraded":"2024-03-18T15:38:06.052007+0000","last_fullsized":"2024-03-18T15:38:06.052007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051936+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:38:06.051936+0000","last_peered":"2024-03-18T15:38:06.051936+0000","last_clean":"2024-03-18T15:38:06.051936+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:38:06.051936+0000","last_undegraded":"2024-03-18T15:38:06.051936+0000","last_fullsized":"2024-03-18T15:38:06.051936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051686+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:38:06.051686+0000","last_peered":"2024-03-18T15:38:06.051686+0000","last_clean":"2024-03-18T15:38:06.051686+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:38:06.051686+0000","last_undegraded":"2024-03-18T15:38:06.051686+0000","last_fullsized":"2024-03-18T15:38:06.051686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051342+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:38:06.051342+0000","last_peered":"2024-03-18T15:38:06.051342+0000","last_clean":"2024-03-18T15:38:06.051342+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:38:06.051342+0000","last_undegraded":"2024-03-18T15:38:06.051342+0000","last_fullsized":"2024-03-18T15:38:06.051342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051246+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:38:06.051246+0000","last_peered":"2024-03-18T15:38:06.051246+0000","last_clean":"2024-03-18T15:38:06.051246+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:38:06.051246+0000","last_undegraded":"2024-03-18T15:38:06.051246+0000","last_fullsized":"2024-03-18T15:38:06.051246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679398+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:38:05.679398+0000","last_peered":"2024-03-18T15:38:05.679398+0000","last_clean":"2024-03-18T15:38:05.679398+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:38:05.679398+0000","last_undegraded":"2024-03-18T15:38:05.679398+0000","last_fullsized":"2024-03-18T15:38:05.679398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051828+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:38:06.051828+0000","last_peered":"2024-03-18T15:38:06.051828+0000","last_clean":"2024-03-18T15:38:06.051828+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:38:06.051828+0000","last_undegraded":"2024-03-18T15:38:06.051828+0000","last_fullsized":"2024-03-18T15:38:06.051828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819672+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:38:05.819672+0000","last_peered":"2024-03-18T15:38:05.819672+0000","last_clean":"2024-03-18T15:38:05.819672+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:38:05.819672+0000","last_undegraded":"2024-03-18T15:38:05.819672+0000","last_fullsized":"2024-03-18T15:38:05.819672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051577+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:38:06.051577+0000","last_peered":"2024-03-18T15:38:06.051577+0000","last_clean":"2024-03-18T15:38:06.051577+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:38:06.051577+0000","last_undegraded":"2024-03-18T15:38:06.051577+0000","last_fullsized":"2024-03-18T15:38:06.051577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.180287+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:38:09.180287+0000","last_peered":"2024-03-18T15:38:09.180287+0000","last_clean":"2024-03-18T15:38:09.180287+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:38:09.180287+0000","last_undegraded":"2024-03-18T15:38:09.180287+0000","last_fullsized":"2024-03-18T15:38:09.180287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679404+0000","last_change":"2024-03-18T15:38:04.692899+0000","last_active":"2024-03-18T15:38:05.679404+0000","last_peered":"2024-03-18T15:38:05.679404+0000","last_clean":"2024-03-18T15:38:05.679404+0000","last_became_active":"2024-03-18T15:38:04.692604+0000","last_became_peered":"2024-03-18T15:38:04.692604+0000","last_unstale":"2024-03-18T15:38:05.679404+0000","last_undegraded":"2024-03-18T15:38:05.679404+0000","last_fullsized":"2024-03-18T15:38:05.679404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:35.353444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679537+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:38:05.679537+0000","last_peered":"2024-03-18T15:38:05.679537+0000","last_clean":"2024-03-18T15:38:05.679537+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:38:05.679537+0000","last_undegraded":"2024-03-18T15:38:05.679537+0000","last_fullsized":"2024-03-18T15:38:05.679537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051610+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:38:06.051610+0000","last_peered":"2024-03-18T15:38:06.051610+0000","last_clean":"2024-03-18T15:38:06.051610+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:38:06.051610+0000","last_undegraded":"2024-03-18T15:38:06.051610+0000","last_fullsized":"2024-03-18T15:38:06.051610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.173187+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:38:08.173187+0000","last_peered":"2024-03-18T15:38:08.173187+0000","last_clean":"2024-03-18T15:38:08.173187+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:38:08.173187+0000","last_undegraded":"2024-03-18T15:38:08.173187+0000","last_fullsized":"2024-03-18T15:38:08.173187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679511+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:38:05.679511+0000","last_peered":"2024-03-18T15:38:05.679511+0000","last_clean":"2024-03-18T15:38:05.679511+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:38:05.679511+0000","last_undegraded":"2024-03-18T15:38:05.679511+0000","last_fullsized":"2024-03-18T15:38:05.679511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824633+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:38:05.824633+0000","last_peered":"2024-03-18T15:38:05.824633+0000","last_clean":"2024-03-18T15:38:05.824633+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:38:05.824633+0000","last_undegraded":"2024-03-18T15:38:05.824633+0000","last_fullsized":"2024-03-18T15:38:05.824633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679950+0000","last_change":"2024-03-18T15:38:04.703055+0000","last_active":"2024-03-18T15:38:05.679950+0000","last_peered":"2024-03-18T15:38:05.679950+0000","last_clean":"2024-03-18T15:38:05.679950+0000","last_became_active":"2024-03-18T15:38:04.702646+0000","last_became_peered":"2024-03-18T15:38:04.702646+0000","last_unstale":"2024-03-18T15:38:05.679950+0000","last_undegraded":"2024-03-18T15:38:05.679950+0000","last_fullsized":"2024-03-18T15:38:05.679950+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:53.447695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165828+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:38:07.165828+0000","last_peered":"2024-03-18T15:38:07.165828+0000","last_clean":"2024-03-18T15:38:07.165828+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:38:07.165828+0000","last_undegraded":"2024-03-18T15:38:07.165828+0000","last_fullsized":"2024-03-18T15:38:07.165828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824410+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:38:05.824410+0000","last_peered":"2024-03-18T15:38:05.824410+0000","last_clean":"2024-03-18T15:38:05.824410+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:38:05.824410+0000","last_undegraded":"2024-03-18T15:38:05.824410+0000","last_fullsized":"2024-03-18T15:38:05.824410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.181058+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:38:09.181058+0000","last_peered":"2024-03-18T15:38:09.181058+0000","last_clean":"2024-03-18T15:38:09.181058+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:38:09.181058+0000","last_undegraded":"2024-03-18T15:38:09.181058+0000","last_fullsized":"2024-03-18T15:38:09.181058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679946+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:38:05.679946+0000","last_peered":"2024-03-18T15:38:05.679946+0000","last_clean":"2024-03-18T15:38:05.679946+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:38:05.679946+0000","last_undegraded":"2024-03-18T15:38:05.679946+0000","last_fullsized":"2024-03-18T15:38:05.679946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.050898+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:38:06.050898+0000","last_peered":"2024-03-18T15:38:06.050898+0000","last_clean":"2024-03-18T15:38:06.050898+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:38:06.050898+0000","last_undegraded":"2024-03-18T15:38:06.050898+0000","last_fullsized":"2024-03-18T15:38:06.050898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679226+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:38:05.679226+0000","last_peered":"2024-03-18T15:38:05.679226+0000","last_clean":"2024-03-18T15:38:05.679226+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:38:05.679226+0000","last_undegraded":"2024-03-18T15:38:05.679226+0000","last_fullsized":"2024-03-18T15:38:05.679226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679214+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:38:05.679214+0000","last_peered":"2024-03-18T15:38:05.679214+0000","last_clean":"2024-03-18T15:38:05.679214+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:38:05.679214+0000","last_undegraded":"2024-03-18T15:38:05.679214+0000","last_fullsized":"2024-03-18T15:38:05.679214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819481+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:38:05.819481+0000","last_peered":"2024-03-18T15:38:05.819481+0000","last_clean":"2024-03-18T15:38:05.819481+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:38:05.819481+0000","last_undegraded":"2024-03-18T15:38:05.819481+0000","last_fullsized":"2024-03-18T15:38:05.819481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.163292+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:38:07.163292+0000","last_peered":"2024-03-18T15:38:07.163292+0000","last_clean":"2024-03-18T15:38:07.163292+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:38:07.163292+0000","last_undegraded":"2024-03-18T15:38:07.163292+0000","last_fullsized":"2024-03-18T15:38:07.163292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679845+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:38:05.679845+0000","last_peered":"2024-03-18T15:38:05.679845+0000","last_clean":"2024-03-18T15:38:05.679845+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:38:05.679845+0000","last_undegraded":"2024-03-18T15:38:05.679845+0000","last_fullsized":"2024-03-18T15:38:05.679845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.164228+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:38:07.164228+0000","last_peered":"2024-03-18T15:38:07.164228+0000","last_clean":"2024-03-18T15:38:07.164228+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:38:07.164228+0000","last_undegraded":"2024-03-18T15:38:07.164228+0000","last_fullsized":"2024-03-18T15:38:07.164228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051551+0000","last_change":"2024-03-18T15:38:04.697178+0000","last_active":"2024-03-18T15:38:06.051551+0000","last_peered":"2024-03-18T15:38:06.051551+0000","last_clean":"2024-03-18T15:38:06.051551+0000","last_became_active":"2024-03-18T15:38:04.696918+0000","last_became_peered":"2024-03-18T15:38:04.696918+0000","last_unstale":"2024-03-18T15:38:06.051551+0000","last_undegraded":"2024-03-18T15:38:06.051551+0000","last_fullsized":"2024-03-18T15:38:06.051551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:45:37.731149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679840+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:38:05.679840+0000","last_peered":"2024-03-18T15:38:05.679840+0000","last_clean":"2024-03-18T15:38:05.679840+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:38:05.679840+0000","last_undegraded":"2024-03-18T15:38:05.679840+0000","last_fullsized":"2024-03-18T15:38:05.679840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679759+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:38:05.679759+0000","last_peered":"2024-03-18T15:38:05.679759+0000","last_clean":"2024-03-18T15:38:05.679759+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:38:05.679759+0000","last_undegraded":"2024-03-18T15:38:05.679759+0000","last_fullsized":"2024-03-18T15:38:05.679759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:10.200004+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:38:10.200004+0000","last_peered":"2024-03-18T15:38:10.200004+0000","last_clean":"2024-03-18T15:38:10.200004+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:38:10.200004+0000","last_undegraded":"2024-03-18T15:38:10.200004+0000","last_fullsized":"2024-03-18T15:38:10.200004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824648+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:38:05.824648+0000","last_peered":"2024-03-18T15:38:05.824648+0000","last_clean":"2024-03-18T15:38:05.824648+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:38:05.824648+0000","last_undegraded":"2024-03-18T15:38:05.824648+0000","last_fullsized":"2024-03-18T15:38:05.824648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.175151+0000","last_change":"2024-03-18T15:38:03.834819+0000","last_active":"2024-03-18T15:38:08.175151+0000","last_peered":"2024-03-18T15:38:08.175151+0000","last_clean":"2024-03-18T15:38:08.175151+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:38:08.175151+0000","last_undegraded":"2024-03-18T15:38:08.175151+0000","last_fullsized":"2024-03-18T15:38:08.175151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:02:11.448328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051269+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:38:06.051269+0000","last_peered":"2024-03-18T15:38:06.051269+0000","last_clean":"2024-03-18T15:38:06.051269+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:38:06.051269+0000","last_undegraded":"2024-03-18T15:38:06.051269+0000","last_fullsized":"2024-03-18T15:38:06.051269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165598+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:38:07.165598+0000","last_peered":"2024-03-18T15:38:07.165598+0000","last_clean":"2024-03-18T15:38:07.165598+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:38:07.165598+0000","last_undegraded":"2024-03-18T15:38:07.165598+0000","last_fullsized":"2024-03-18T15:38:07.165598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052145+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:38:06.052145+0000","last_peered":"2024-03-18T15:38:06.052145+0000","last_clean":"2024-03-18T15:38:06.052145+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:38:06.052145+0000","last_undegraded":"2024-03-18T15:38:06.052145+0000","last_fullsized":"2024-03-18T15:38:06.052145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.174803+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:38:08.174803+0000","last_peered":"2024-03-18T15:38:08.174803+0000","last_clean":"2024-03-18T15:38:08.174803+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:38:08.174803+0000","last_undegraded":"2024-03-18T15:38:08.174803+0000","last_fullsized":"2024-03-18T15:38:08.174803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051335+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:38:06.051335+0000","last_peered":"2024-03-18T15:38:06.051335+0000","last_clean":"2024-03-18T15:38:06.051335+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:38:06.051335+0000","last_undegraded":"2024-03-18T15:38:06.051335+0000","last_fullsized":"2024-03-18T15:38:06.051335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.176484+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:38:08.176484+0000","last_peered":"2024-03-18T15:38:08.176484+0000","last_clean":"2024-03-18T15:38:08.176484+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:38:08.176484+0000","last_undegraded":"2024-03-18T15:38:08.176484+0000","last_fullsized":"2024-03-18T15:38:08.176484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679551+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:38:05.679551+0000","last_peered":"2024-03-18T15:38:05.679551+0000","last_clean":"2024-03-18T15:38:05.679551+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:38:05.679551+0000","last_undegraded":"2024-03-18T15:38:05.679551+0000","last_fullsized":"2024-03-18T15:38:05.679551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824640+0000","last_change":"2024-03-18T15:38:03.831562+0000","last_active":"2024-03-18T15:38:05.824640+0000","last_peered":"2024-03-18T15:38:05.824640+0000","last_clean":"2024-03-18T15:38:05.824640+0000","last_became_active":"2024-03-18T15:38:03.831204+0000","last_became_peered":"2024-03-18T15:38:03.831204+0000","last_unstale":"2024-03-18T15:38:05.824640+0000","last_undegraded":"2024-03-18T15:38:05.824640+0000","last_fullsized":"2024-03-18T15:38:05.824640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:22:14.867492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051484+0000","last_change":"2024-03-18T15:38:04.693705+0000","last_active":"2024-03-18T15:38:06.051484+0000","last_peered":"2024-03-18T15:38:06.051484+0000","last_clean":"2024-03-18T15:38:06.051484+0000","last_became_active":"2024-03-18T15:38:04.693283+0000","last_became_peered":"2024-03-18T15:38:04.693283+0000","last_unstale":"2024-03-18T15:38:06.051484+0000","last_undegraded":"2024-03-18T15:38:06.051484+0000","last_fullsized":"2024-03-18T15:38:06.051484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:52:56.238305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.819572+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:38:05.819572+0000","last_peered":"2024-03-18T15:38:05.819572+0000","last_clean":"2024-03-18T15:38:05.819572+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:38:05.819572+0000","last_undegraded":"2024-03-18T15:38:05.819572+0000","last_fullsized":"2024-03-18T15:38:05.819572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052937+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:38:06.052937+0000","last_peered":"2024-03-18T15:38:06.052937+0000","last_clean":"2024-03-18T15:38:06.052937+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:38:06.052937+0000","last_undegraded":"2024-03-18T15:38:06.052937+0000","last_fullsized":"2024-03-18T15:38:06.052937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051428+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:38:06.051428+0000","last_peered":"2024-03-18T15:38:06.051428+0000","last_clean":"2024-03-18T15:38:06.051428+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:38:06.051428+0000","last_undegraded":"2024-03-18T15:38:06.051428+0000","last_fullsized":"2024-03-18T15:38:06.051428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052788+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:38:06.052788+0000","last_peered":"2024-03-18T15:38:06.052788+0000","last_clean":"2024-03-18T15:38:06.052788+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:38:06.052788+0000","last_undegraded":"2024-03-18T15:38:06.052788+0000","last_fullsized":"2024-03-18T15:38:06.052788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679489+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:38:05.679489+0000","last_peered":"2024-03-18T15:38:05.679489+0000","last_clean":"2024-03-18T15:38:05.679489+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:38:05.679489+0000","last_undegraded":"2024-03-18T15:38:05.679489+0000","last_fullsized":"2024-03-18T15:38:05.679489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.162720+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:38:07.162720+0000","last_peered":"2024-03-18T15:38:07.162720+0000","last_clean":"2024-03-18T15:38:07.162720+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:38:07.162720+0000","last_undegraded":"2024-03-18T15:38:07.162720+0000","last_fullsized":"2024-03-18T15:38:07.162720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679536+0000","last_change":"2024-03-18T15:38:04.689717+0000","last_active":"2024-03-18T15:38:05.679536+0000","last_peered":"2024-03-18T15:38:05.679536+0000","last_clean":"2024-03-18T15:38:05.679536+0000","last_became_active":"2024-03-18T15:38:04.689352+0000","last_became_peered":"2024-03-18T15:38:04.689352+0000","last_unstale":"2024-03-18T15:38:05.679536+0000","last_undegraded":"2024-03-18T15:38:05.679536+0000","last_fullsized":"2024-03-18T15:38:05.679536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:58.058080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165059+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:38:07.165059+0000","last_peered":"2024-03-18T15:38:07.165059+0000","last_clean":"2024-03-18T15:38:07.165059+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:38:07.165059+0000","last_undegraded":"2024-03-18T15:38:07.165059+0000","last_fullsized":"2024-03-18T15:38:07.165059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051535+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:38:06.051535+0000","last_peered":"2024-03-18T15:38:06.051535+0000","last_clean":"2024-03-18T15:38:06.051535+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:38:06.051535+0000","last_undegraded":"2024-03-18T15:38:06.051535+0000","last_fullsized":"2024-03-18T15:38:06.051535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.050851+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:38:06.050851+0000","last_peered":"2024-03-18T15:38:06.050851+0000","last_clean":"2024-03-18T15:38:06.050851+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:38:06.050851+0000","last_undegraded":"2024-03-18T15:38:06.050851+0000","last_fullsized":"2024-03-18T15:38:06.050851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.173915+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:38:08.173915+0000","last_peered":"2024-03-18T15:38:08.173915+0000","last_clean":"2024-03-18T15:38:08.173915+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:38:08.173915+0000","last_undegraded":"2024-03-18T15:38:08.173915+0000","last_fullsized":"2024-03-18T15:38:08.173915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824744+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:38:05.824744+0000","last_peered":"2024-03-18T15:38:05.824744+0000","last_clean":"2024-03-18T15:38:05.824744+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:38:05.824744+0000","last_undegraded":"2024-03-18T15:38:05.824744+0000","last_fullsized":"2024-03-18T15:38:05.824744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051637+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:38:06.051637+0000","last_peered":"2024-03-18T15:38:06.051637+0000","last_clean":"2024-03-18T15:38:06.051637+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:38:06.051637+0000","last_undegraded":"2024-03-18T15:38:06.051637+0000","last_fullsized":"2024-03-18T15:38:06.051637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:09.180964+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:38:09.180964+0000","last_peered":"2024-03-18T15:38:09.180964+0000","last_clean":"2024-03-18T15:38:09.180964+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:38:09.180964+0000","last_undegraded":"2024-03-18T15:38:09.180964+0000","last_fullsized":"2024-03-18T15:38:09.180964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679531+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:38:05.679531+0000","last_peered":"2024-03-18T15:38:05.679531+0000","last_clean":"2024-03-18T15:38:05.679531+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:38:05.679531+0000","last_undegraded":"2024-03-18T15:38:05.679531+0000","last_fullsized":"2024-03-18T15:38:05.679531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.163439+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:38:07.163439+0000","last_peered":"2024-03-18T15:38:07.163439+0000","last_clean":"2024-03-18T15:38:07.163439+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:38:07.163439+0000","last_undegraded":"2024-03-18T15:38:07.163439+0000","last_fullsized":"2024-03-18T15:38:07.163439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051189+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:38:06.051189+0000","last_peered":"2024-03-18T15:38:06.051189+0000","last_clean":"2024-03-18T15:38:06.051189+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:38:06.051189+0000","last_undegraded":"2024-03-18T15:38:06.051189+0000","last_fullsized":"2024-03-18T15:38:06.051189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.162336+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:38:07.162336+0000","last_peered":"2024-03-18T15:38:07.162336+0000","last_clean":"2024-03-18T15:38:07.162336+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:38:07.162336+0000","last_undegraded":"2024-03-18T15:38:07.162336+0000","last_fullsized":"2024-03-18T15:38:07.162336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.824372+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:38:05.824372+0000","last_peered":"2024-03-18T15:38:05.824372+0000","last_clean":"2024-03-18T15:38:05.824372+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:38:05.824372+0000","last_undegraded":"2024-03-18T15:38:05.824372+0000","last_fullsized":"2024-03-18T15:38:05.824372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165304+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:38:07.165304+0000","last_peered":"2024-03-18T15:38:07.165304+0000","last_clean":"2024-03-18T15:38:07.165304+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:38:07.165304+0000","last_undegraded":"2024-03-18T15:38:07.165304+0000","last_fullsized":"2024-03-18T15:38:07.165304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051077+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:38:06.051077+0000","last_peered":"2024-03-18T15:38:06.051077+0000","last_clean":"2024-03-18T15:38:06.051077+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:38:06.051077+0000","last_undegraded":"2024-03-18T15:38:06.051077+0000","last_fullsized":"2024-03-18T15:38:06.051077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051329+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:38:06.051329+0000","last_peered":"2024-03-18T15:38:06.051329+0000","last_clean":"2024-03-18T15:38:06.051329+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:38:06.051329+0000","last_undegraded":"2024-03-18T15:38:06.051329+0000","last_fullsized":"2024-03-18T15:38:06.051329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679374+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:38:05.679374+0000","last_peered":"2024-03-18T15:38:05.679374+0000","last_clean":"2024-03-18T15:38:05.679374+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:38:05.679374+0000","last_undegraded":"2024-03-18T15:38:05.679374+0000","last_fullsized":"2024-03-18T15:38:05.679374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:07.165484+0000","last_change":"2024-03-18T15:38:03.833151+0000","last_active":"2024-03-18T15:38:07.165484+0000","last_peered":"2024-03-18T15:38:07.165484+0000","last_clean":"2024-03-18T15:38:07.165484+0000","last_became_active":"2024-03-18T15:38:03.832775+0000","last_became_peered":"2024-03-18T15:38:03.832775+0000","last_unstale":"2024-03-18T15:38:07.165484+0000","last_undegraded":"2024-03-18T15:38:07.165484+0000","last_fullsized":"2024-03-18T15:38:07.165484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:13.936881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051550+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:38:06.051550+0000","last_peered":"2024-03-18T15:38:06.051550+0000","last_clean":"2024-03-18T15:38:06.051550+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:38:06.051550+0000","last_undegraded":"2024-03-18T15:38:06.051550+0000","last_fullsized":"2024-03-18T15:38:06.051550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:08.174761+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:38:08.174761+0000","last_peered":"2024-03-18T15:38:08.174761+0000","last_clean":"2024-03-18T15:38:08.174761+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:38:08.174761+0000","last_undegraded":"2024-03-18T15:38:08.174761+0000","last_fullsized":"2024-03-18T15:38:08.174761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.052115+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:38:06.052115+0000","last_peered":"2024-03-18T15:38:06.052115+0000","last_clean":"2024-03-18T15:38:06.052115+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:38:06.052115+0000","last_undegraded":"2024-03-18T15:38:06.052115+0000","last_fullsized":"2024-03-18T15:38:06.052115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679556+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:38:05.679556+0000","last_peered":"2024-03-18T15:38:05.679556+0000","last_clean":"2024-03-18T15:38:05.679556+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:38:05.679556+0000","last_undegraded":"2024-03-18T15:38:05.679556+0000","last_fullsized":"2024-03-18T15:38:05.679556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051399+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:38:06.051399+0000","last_peered":"2024-03-18T15:38:06.051399+0000","last_clean":"2024-03-18T15:38:06.051399+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:38:06.051399+0000","last_undegraded":"2024-03-18T15:38:06.051399+0000","last_fullsized":"2024-03-18T15:38:06.051399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051687+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:38:06.051687+0000","last_peered":"2024-03-18T15:38:06.051687+0000","last_clean":"2024-03-18T15:38:06.051687+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:38:06.051687+0000","last_undegraded":"2024-03-18T15:38:06.051687+0000","last_fullsized":"2024-03-18T15:38:06.051687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051815+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:38:06.051815+0000","last_peered":"2024-03-18T15:38:06.051815+0000","last_clean":"2024-03-18T15:38:06.051815+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:38:06.051815+0000","last_undegraded":"2024-03-18T15:38:06.051815+0000","last_fullsized":"2024-03-18T15:38:06.051815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051102+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:38:06.051102+0000","last_peered":"2024-03-18T15:38:06.051102+0000","last_clean":"2024-03-18T15:38:06.051102+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:38:06.051102+0000","last_undegraded":"2024-03-18T15:38:06.051102+0000","last_fullsized":"2024-03-18T15:38:06.051102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051246+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:38:06.051246+0000","last_peered":"2024-03-18T15:38:06.051246+0000","last_clean":"2024-03-18T15:38:06.051246+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:38:06.051246+0000","last_undegraded":"2024-03-18T15:38:06.051246+0000","last_fullsized":"2024-03-18T15:38:06.051246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051279+0000","last_change":"2024-03-18T15:38:04.684780+0000","last_active":"2024-03-18T15:38:06.051279+0000","last_peered":"2024-03-18T15:38:06.051279+0000","last_clean":"2024-03-18T15:38:06.051279+0000","last_became_active":"2024-03-18T15:38:04.684376+0000","last_became_peered":"2024-03-18T15:38:04.684376+0000","last_unstale":"2024-03-18T15:38:06.051279+0000","last_undegraded":"2024-03-18T15:38:06.051279+0000","last_fullsized":"2024-03-18T15:38:06.051279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:47:40.082093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051589+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:38:06.051589+0000","last_peered":"2024-03-18T15:38:06.051589+0000","last_clean":"2024-03-18T15:38:06.051589+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:38:06.051589+0000","last_undegraded":"2024-03-18T15:38:06.051589+0000","last_fullsized":"2024-03-18T15:38:06.051589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051485+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:38:06.051485+0000","last_peered":"2024-03-18T15:38:06.051485+0000","last_clean":"2024-03-18T15:38:06.051485+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:38:06.051485+0000","last_undegraded":"2024-03-18T15:38:06.051485+0000","last_fullsized":"2024-03-18T15:38:06.051485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051421+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:38:06.051421+0000","last_peered":"2024-03-18T15:38:06.051421+0000","last_clean":"2024-03-18T15:38:06.051421+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:38:06.051421+0000","last_undegraded":"2024-03-18T15:38:06.051421+0000","last_fullsized":"2024-03-18T15:38:06.051421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051708+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:38:06.051708+0000","last_peered":"2024-03-18T15:38:06.051708+0000","last_clean":"2024-03-18T15:38:06.051708+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:38:06.051708+0000","last_undegraded":"2024-03-18T15:38:06.051708+0000","last_fullsized":"2024-03-18T15:38:06.051708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679510+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:38:05.679510+0000","last_peered":"2024-03-18T15:38:05.679510+0000","last_clean":"2024-03-18T15:38:05.679510+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:38:05.679510+0000","last_undegraded":"2024-03-18T15:38:05.679510+0000","last_fullsized":"2024-03-18T15:38:05.679510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679580+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:38:05.679580+0000","last_peered":"2024-03-18T15:38:05.679580+0000","last_clean":"2024-03-18T15:38:05.679580+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:38:05.679580+0000","last_undegraded":"2024-03-18T15:38:05.679580+0000","last_fullsized":"2024-03-18T15:38:05.679580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679068+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:38:05.679068+0000","last_peered":"2024-03-18T15:38:05.679068+0000","last_clean":"2024-03-18T15:38:05.679068+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:38:05.679068+0000","last_undegraded":"2024-03-18T15:38:05.679068+0000","last_fullsized":"2024-03-18T15:38:05.679068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:05.679641+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:38:05.679641+0000","last_peered":"2024-03-18T15:38:05.679641+0000","last_clean":"2024-03-18T15:38:05.679641+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:38:05.679641+0000","last_undegraded":"2024-03-18T15:38:05.679641+0000","last_fullsized":"2024-03-18T15:38:05.679641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051726+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:38:06.051726+0000","last_peered":"2024-03-18T15:38:06.051726+0000","last_clean":"2024-03-18T15:38:06.051726+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:38:06.051726+0000","last_undegraded":"2024-03-18T15:38:06.051726+0000","last_fullsized":"2024-03-18T15:38:06.051726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-18T15:38:06.051267+0000","last_change":"2024-03-18T15:38:04.699503+0000","last_active":"2024-03-18T15:38:06.051267+0000","last_peered":"2024-03-18T15:38:06.051267+0000","last_clean":"2024-03-18T15:38:06.051267+0000","last_became_active":"2024-03-18T15:38:04.699163+0000","last_became_peered":"2024-03-18T15:38:04.699163+0000","last_unstale":"2024-03-18T15:38:06.051267+0000","last_undegraded":"2024-03-18T15:38:06.051267+0000","last_fullsized":"2024-03-18T15:38:06.051267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:16:18.868944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":24505,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":54082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":913408,"data_stored":434445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1056131,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":1035175,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":464492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":445977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":443119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":443659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1034859,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":22647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:38:12.223 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T15:38:12.224 INFO:tasks.cephadm:Setting up client nodes... 2024-03-18T15:38:12.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-18T15:38:12.495 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:12.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:12 smithi137 ceph-mon[35868]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:12.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:12 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2719306827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:12.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:12 smithi137 ceph-mon[35868]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s wr, 19 op/s 2024-03-18T15:38:12.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:12 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1469647122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:12.616 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:12 smithi096 ceph-mon[31930]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:12.616 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:12 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2719306827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:12.616 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:12 smithi096 ceph-mon[31930]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s wr, 19 op/s 2024-03-18T15:38:12.616 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:12 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1469647122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:12.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:12 smithi193 ceph-mon[36702]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:38:12.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:12 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2719306827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:38:12.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:12 smithi193 ceph-mon[36702]: pgmap v291: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s wr, 19 op/s 2024-03-18T15:38:12.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:12 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1469647122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:13.518 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:13 smithi096 ceph-mon[31930]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:13.518 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:13 smithi096 ceph-mon[31930]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:13.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:13 smithi137 ceph-mon[35868]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:13.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:13 smithi137 ceph-mon[35868]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:13.667 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:13 smithi193 ceph-mon[36702]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:13.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:13 smithi193 ceph-mon[36702]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 -- 172.21.15.96:0/526543021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40101220 msgr2=0x7ffb40101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/526543021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40101220 0x7ffb40101640 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffb300099c0 tx=0x7ffb30009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 -- 172.21.15.96:0/526543021 shutdown_connections 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/526543021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb40109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/526543021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40101220 0x7ffb40101640 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/526543021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40102420 0x7ffb40102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 -- 172.21.15.96:0/526543021 >> 172.21.15.96:0/526543021 conn(0x7ffb400fc7c0 msgr2=0x7ffb400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 -- 172.21.15.96:0/526543021 shutdown_connections 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.123+0000 7ffb44ee9700 1 -- 172.21.15.96:0/526543021 wait complete. 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 Processor -- start 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 -- start start 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 0x7ffb4019d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb4019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4010ce10 con 0x7ffb40102420 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4010cc90 con 0x7ffb40101220 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb44ee9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4010cf90 con 0x7ffb40102dc0 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 0x7ffb4019d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41334/0 (socket says 172.21.15.96:41334) 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 0x7ffb4019d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54382/0 (socket says 172.21.15.96:54382) 2024-03-18T15:38:14.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3e59c700 1 -- 172.21.15.96:0/974477915 learned_addr learned my addr 172.21.15.96:0/974477915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.124+0000 7ffb3ed9d700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb4019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 -- 172.21.15.96:0/974477915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 msgr2=0x7ffb4019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb4019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 -- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 msgr2=0x7ffb4019d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 0x7ffb4019d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 -- 172.21.15.96:0/974477915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb401a00f0 con 0x7ffb40101220 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3ed9d700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb4019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb3e59c700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000bf60 tx=0x7ffb3000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30009e60 con 0x7ffb40101220 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb401a0380 con 0x7ffb40101220 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb30003e30 con 0x7ffb40101220 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000fdd0 con 0x7ffb40101220 2024-03-18T15:38:14.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.125+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a08b0 con 0x7ffb40101220 2024-03-18T15:38:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.126+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb400507e0 con 0x7ffb40101220 2024-03-18T15:38:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.127+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ffb3000f4f0 con 0x7ffb40101220 2024-03-18T15:38:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.128+0000 7ffb2f7fe700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 0x7ffb280a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.128+0000 7ffb3dd9b700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 0x7ffb280a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:14.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.128+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffb300cf8b0 con 0x7ffb40101220 2024-03-18T15:38:14.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.128+0000 7ffb3dd9b700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 0x7ffb280a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb4006bad0 tx=0x7ffb34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:14.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.136+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ffb30013070 con 0x7ffb40101220 2024-03-18T15:38:14.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.413+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ffb4002db70 con 0x7ffb40101220 2024-03-18T15:38:14.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:14 smithi096 ceph-mon[31930]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2024-03-18T15:38:14.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.420+0000 7ffb2f7fe700 1 -- 172.21.15.96:0/974477915 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ffb3009ca30 con 0x7ffb40101220 2024-03-18T15:38:14.421 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-03-18T15:38:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQDmX/hlVt7GGBAAKMsWsUdEtc918mxfVN28aw== 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 msgr2=0x7ffb280a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 0x7ffb280a4e90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffb4006bad0 tx=0x7ffb34009260 comp rx=0 tx=0).stop 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 msgr2=0x7ffb4006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000bf60 tx=0x7ffb3000bf90 comp rx=0 tx=0).stop 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 shutdown_connections 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffb280a29d0 0x7ffb280a4e90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb40102dc0 0x7ffb4019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb40101220 0x7ffb4006b570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 --2- 172.21.15.96:0/974477915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb40102420 0x7ffb4019d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:14.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 >> 172.21.15.96:0/974477915 conn(0x7ffb400fc7c0 msgr2=0x7ffb40105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:14.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 shutdown_connections 2024-03-18T15:38:14.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:14.422+0000 7ffb44ee9700 1 -- 172.21.15.96:0/974477915 wait complete. 2024-03-18T15:38:14.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:14 smithi193 ceph-mon[36702]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2024-03-18T15:38:14.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:14 smithi137 ceph-mon[35868]: pgmap v292: 97 pgs: 97 active+clean; 601 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2024-03-18T15:38:14.917 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:38:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:38:14.526+0000 7f4361e84700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-18T15:38:15.072 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:38:15.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-18T15:38:15.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T15:38:15.116 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T15:38:15.116 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-18T15:38:15.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-18T15:38:15.394 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:15.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:15 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/974477915' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:15 smithi096 ceph-mon[31930]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:15 smithi096 ceph-mon[31930]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T15:38:15.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:15 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/974477915' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:15 smithi137 ceph-mon[35868]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:15 smithi137 ceph-mon[35868]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T15:38:15.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:15 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/974477915' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:15 smithi193 ceph-mon[36702]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-18T15:38:15.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:15 smithi193 ceph-mon[36702]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-18T15:38:16.595 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:16 smithi096 ceph-mon[31930]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-18T15:38:16.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:16 smithi137 ceph-mon[35868]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-18T15:38:16.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:16 smithi193 ceph-mon[36702]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 -- 172.21.15.96:0/2573456523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102ec0 msgr2=0x7fac2c106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 --2- 172.21.15.96:0/2573456523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102ec0 0x7fac2c106f10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0099c0 tx=0x7fac1c009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 -- 172.21.15.96:0/2573456523 shutdown_connections 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 --2- 172.21.15.96:0/2573456523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c1075f0 0x7fac2c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 --2- 172.21.15.96:0/2573456523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c102510 0x7fac2c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 --2- 172.21.15.96:0/2573456523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102ec0 0x7fac2c106f10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 -- 172.21.15.96:0/2573456523 >> 172.21.15.96:0/2573456523 conn(0x7fac2c0fdd80 msgr2=0x7fac2c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 -- 172.21.15.96:0/2573456523 shutdown_connections 2024-03-18T15:38:17.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.106+0000 7fac329d9700 1 -- 172.21.15.96:0/2573456523 wait complete. 2024-03-18T15:38:17.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 Processor -- start 2024-03-18T15:38:17.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 -- start start 2024-03-18T15:38:17.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c102ec0 0x7fac2c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c1075f0 0x7fac2c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c1125b0 con 0x7fac2c102510 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c112430 con 0x7fac2c1075f0 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac329d9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c112730 con 0x7fac2c102ec0 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54398/0 (socket says 172.21.15.96:54398) 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 -- 172.21.15.96:0/3517257018 learned_addr learned my addr 172.21.15.96:0/3517257018 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2b7fe700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c102ec0 0x7fac2c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac30f76700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c1075f0 0x7fac2c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 -- 172.21.15.96:0/3517257018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c102ec0 msgr2=0x7fac2c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c102ec0 0x7fac2c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 -- 172.21.15.96:0/3517257018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c1075f0 msgr2=0x7fac2c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:17.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c1075f0 0x7fac2c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.107+0000 7fac2bfff700 1 -- 172.21.15.96:0/3517257018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c199010 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac2bfff700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d980 tx=0x7fac1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14016070 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac14004d10 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c199300 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14005670 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.108+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1a55c0 con 0x7fac2c102510 2024-03-18T15:38:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.109+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c068f90 con 0x7fac2c102510 2024-03-18T15:38:17.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.110+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fac14004750 con 0x7fac2c102510 2024-03-18T15:38:17.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.110+0000 7fac297fa700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 0x7fac180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:17.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.110+0000 7fac2b7fe700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 0x7fac180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:17.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.110+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fac140ce800 con 0x7fac2c102510 2024-03-18T15:38:17.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.111+0000 7fac2b7fe700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 0x7fac180a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:17.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.115+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fac1401f020 con 0x7fac2c102510 2024-03-18T15:38:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.368+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fac2c02db70 con 0x7fac2c102510 2024-03-18T15:38:17.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.373+0000 7fac297fa700 1 -- 172.21.15.96:0/3517257018 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+262659 (secure 0 0 0) 0x7fac14022350 con 0x7fac2c102510 2024-03-18T15:38:17.378 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:17.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 msgr2=0x7fac180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:17.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 0x7fac180a4de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000c00 tx=0x7fac1c005fb0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 msgr2=0x7fac2c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:17.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d980 tx=0x7fac1400dc90 comp rx=0 tx=0).stop 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 shutdown_connections 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fac180a2920 0x7fac180a4de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac2c1075f0 0x7fac2c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac2c102510 0x7fac2c19e6f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 --2- 172.21.15.96:0/3517257018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac2c102ec0 0x7fac2c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 >> 172.21.15.96:0/3517257018 conn(0x7fac2c0fdd80 msgr2=0x7fac2c100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 shutdown_connections 2024-03-18T15:38:17.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:17.376+0000 7fac329d9700 1 -- 172.21.15.96:0/3517257018 wait complete. 2024-03-18T15:38:17.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:17 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3517257018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T15:38:17.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:17 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3517257018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T15:38:17.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:17 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3517257018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-18T15:38:18.082 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3036369138},{"type":"v1","addr":"172.21.15.96:6801","nonce":3036369138}]},"active_addr":"172.21.15.96:6801/3036369138","active_change":"2024-03-18T15:31:01.361112+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.96:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":72482423}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3295206201}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1723493596}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3683302653}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3463232293}]}]} 2024-03-18T15:38:18.087 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-18T15:38:18.087 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-18T15:38:18.087 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T15:38:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:18.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:18 smithi096 ceph-mon[31930]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-18T15:38:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:18 smithi137 ceph-mon[35868]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-18T15:38:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:18 smithi193 ceph-mon[36702]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-03-18T15:38:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 -- 172.21.15.96:0/3389850685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101e10 msgr2=0x7f6b38105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:19.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/3389850685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101e10 0x7f6b38105dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6b280099c0 tx=0x7f6b28009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 -- 172.21.15.96:0/3389850685 shutdown_connections 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/3389850685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 0x7f6b3810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/3389850685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101460 0x7f6b38101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/3389850685 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101e10 0x7f6b38105dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 -- 172.21.15.96:0/3389850685 >> 172.21.15.96:0/3389850685 conn(0x7f6b38077220 msgr2=0x7f6b38077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 -- 172.21.15.96:0/3389850685 shutdown_connections 2024-03-18T15:38:19.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.975+0000 7f6b3fd25700 1 -- 172.21.15.96:0/3389850685 wait complete. 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 Processor -- start 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 -- start start 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 0x7f6b38199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 0x7f6b381a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b381114e0 con 0x7f6b38101460 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38111360 con 0x7f6b38101e10 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3fd25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38111660 con 0x7f6b381064b0 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.976+0000 7f6b3dac1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54420/0 (socket says 172.21.15.96:54420) 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 -- 172.21.15.96:0/1547950338 learned_addr learned my addr 172.21.15.96:0/1547950338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3d2c0700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 0x7f6b38199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3e2c2700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 0x7f6b381a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 -- 172.21.15.96:0/1547950338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 msgr2=0x7f6b381a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 0x7f6b381a4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 -- 172.21.15.96:0/1547950338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 msgr2=0x7f6b38199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 0x7f6b38199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 -- 172.21.15.96:0/1547950338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b381a4ec0 con 0x7f6b38101460 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3d2c0700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 0x7f6b38199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3dac1700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d950 tx=0x7f6b3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34016070 con 0x7f6b38101460 2024-03-18T15:38:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b3400fd70 con 0x7f6b38101460 2024-03-18T15:38:19.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b38197750 con 0x7f6b38101460 2024-03-18T15:38:19.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400e610 con 0x7f6b38101460 2024-03-18T15:38:19.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.977+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b38197cb0 con 0x7f6b38101460 2024-03-18T15:38:19.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.978+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b38068f90 con 0x7f6b38101460 2024-03-18T15:38:19.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.980+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6b34019070 con 0x7f6b38101460 2024-03-18T15:38:19.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.981+0000 7f6b2effd700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 0x7f6b240a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:19.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.981+0000 7f6b3d2c0700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 0x7f6b240a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:19.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.981+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6b340ce450 con 0x7f6b38101460 2024-03-18T15:38:19.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.981+0000 7f6b3d2c0700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 0x7f6b240a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000c00 tx=0x7f6b28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:19.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:19.985+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f6b3409b540 con 0x7f6b38101460 2024-03-18T15:38:20.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.184+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b3802db70 con 0x7f6b38101460 2024-03-18T15:38:20.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.185+0000 7f6b2effd700 1 -- 172.21.15.96:0/1547950338 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22171 (secure 0 0 0) 0x7f6b3409ae40 con 0x7f6b38101460 2024-03-18T15:38:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:20.186 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":86,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:38:05.670715+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 msgr2=0x7f6b240a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 0x7f6b240a4e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000c00 tx=0x7f6b28005fb0 comp rx=0 tx=0).stop 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 msgr2=0x7f6b38199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400d950 tx=0x7f6b3400dc60 comp rx=0 tx=0).stop 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 shutdown_connections 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6b240a29a0 0x7f6b240a4e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381064b0 0x7f6b381a4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b38101460 0x7f6b38199900 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 --2- 172.21.15.96:0/1547950338 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b38101e10 0x7f6b38199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 >> 172.21.15.96:0/1547950338 conn(0x7f6b38077220 msgr2=0x7f6b380ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 shutdown_connections 2024-03-18T15:38:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:20.187+0000 7f6b3fd25700 1 -- 172.21.15.96:0/1547950338 wait complete. 2024-03-18T15:38:20.714 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:20 smithi096 ceph-mon[31930]: pgmap v295: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-18T15:38:20.714 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:20 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1547950338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:20 smithi137 ceph-mon[35868]: pgmap v295: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-18T15:38:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:20 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1547950338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:20 smithi193 ceph-mon[36702]: pgmap v295: 97 pgs: 97 active+clean; 625 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.8 KiB/s wr, 25 op/s 2024-03-18T15:38:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:20 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1547950338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:21.093 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-18T15:38:21.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-18T15:38:21.333 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:22.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:22 smithi096 ceph-mon[31930]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-18T15:38:22.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:22 smithi137 ceph-mon[35868]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-18T15:38:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:22 smithi193 ceph-mon[36702]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-18T15:38:23.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 -- 172.21.15.96:0/3104792776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 msgr2=0x7fb824102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 --2- 172.21.15.96:0/3104792776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb824102010 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb8140099c0 tx=0x7fb814009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 -- 172.21.15.96:0/3104792776 shutdown_connections 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 --2- 172.21.15.96:0/3104792776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb824106550 0x7fb82410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 --2- 172.21.15.96:0/3104792776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb824102010 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 --2- 172.21.15.96:0/3104792776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8241025e0 0x7fb824105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 -- 172.21.15.96:0/3104792776 >> 172.21.15.96:0/3104792776 conn(0x7fb8240777d0 msgr2=0x7fb8240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 -- 172.21.15.96:0/3104792776 shutdown_connections 2024-03-18T15:38:23.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.239+0000 7fb829183700 1 -- 172.21.15.96:0/3104792776 wait complete. 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 Processor -- start 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 -- start start 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 0x7fb82419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb824106550 0x7fb824197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb824111530 con 0x7fb824101c30 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241113b0 con 0x7fb8241025e0 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb829183700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb8241116b0 con 0x7fb824106550 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb822d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb822d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54444/0 (socket says 172.21.15.96:54444) 2024-03-18T15:38:23.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb822d9d700 1 -- 172.21.15.96:0/1795017891 learned_addr learned my addr 172.21.15.96:0/1795017891 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb82359e700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb824106550 0x7fb824197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb82259c700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 0x7fb82419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.240+0000 7fb822d9d700 1 -- 172.21.15.96:0/1795017891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb824106550 msgr2=0x7fb824197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb822d9d700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb824106550 0x7fb824197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb822d9d700 1 -- 172.21.15.96:0/1795017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 msgr2=0x7fb82419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb822d9d700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 0x7fb82419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb822d9d700 1 -- 172.21.15.96:0/1795017891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb824197ff0 con 0x7fb824101c30 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb82259c700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 0x7fb82419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb822d9d700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb814006010 tx=0x7fb814003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814009e60 con 0x7fb824101c30 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb814003680 con 0x7fb824101c30 2024-03-18T15:38:23.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb824198280 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81400f6c0 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.241+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241008c0 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.242+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8241029e0 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.243+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb814019770 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.243+0000 7fb80bfff700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 0x7fb80c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.244+0000 7fb82259c700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 0x7fb80c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.244+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb8140d1a30 con 0x7fb824101c30 2024-03-18T15:38:23.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.244+0000 7fb82259c700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 0x7fb80c0a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb824107f40 tx=0x7fb8180073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:23.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.248+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb814013070 con 0x7fb824101c30 2024-03-18T15:38:23.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.444+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb824198cf0 con 0x7fb824101c30 2024-03-18T15:38:23.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.446+0000 7fb80bfff700 1 -- 172.21.15.96:0/1795017891 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22171 (secure 0 0 0) 0x7fb814022370 con 0x7fb824101c30 2024-03-18T15:38:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:38:23.447 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":86,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:38:05.670715+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 msgr2=0x7fb80c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 0x7fb80c0a4de0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb824107f40 tx=0x7fb8180073d0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 msgr2=0x7fb82419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb814006010 tx=0x7fb814003e00 comp rx=0 tx=0).stop 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 shutdown_connections 2024-03-18T15:38:23.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb80c0a2920 0x7fb80c0a4de0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb824106550 0x7fb824197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101c30 0x7fb82419d640 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 --2- 172.21.15.96:0/1795017891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8241025e0 0x7fb82419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:23.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 >> 172.21.15.96:0/1795017891 conn(0x7fb8240777d0 msgr2=0x7fb8240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:23.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 shutdown_connections 2024-03-18T15:38:23.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:23.448+0000 7fb829183700 1 -- 172.21.15.96:0/1795017891 wait complete. 2024-03-18T15:38:23.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:23 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1795017891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:23 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1795017891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:23 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1795017891' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:38:24.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-18T15:38:24.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-18T15:38:24.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-18T15:38:24.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-18T15:38:24.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-18T15:38:24.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-18T15:38:24.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-18T15:38:24.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-18T15:38:24.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-18T15:38:24.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-18T15:38:24.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-18T15:38:24.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-18T15:38:24.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:24 smithi137 ceph-mon[35868]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-03-18T15:38:24.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:24 smithi193 ceph-mon[36702]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-03-18T15:38:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:24 smithi096 ceph-mon[31930]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 18 op/s 2024-03-18T15:38:24.954 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:25.024 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:26.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:26 smithi137 ceph-mon[35868]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-03-18T15:38:26.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:26 smithi193 ceph-mon[36702]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-03-18T15:38:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:26 smithi096 ceph-mon[31930]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-03-18T15:38:27.046 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.047 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.047 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.047 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.048 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.051 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.483 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.483 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:27.535 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:28.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:28 smithi096 ceph-mon[31930]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T15:38:28.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:28 smithi137 ceph-mon[35868]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T15:38:28.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:28 smithi193 ceph-mon[36702]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-03-18T15:38:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:30 smithi096 ceph-mon[31930]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:38:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:30 smithi137 ceph-mon[35868]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:38:30.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:30 smithi193 ceph-mon[36702]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:38:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:32 smithi137 ceph-mon[35868]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T15:38:32.897 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:32 smithi096 ceph-mon[31930]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T15:38:32.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:32 smithi193 ceph-mon[36702]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-18T15:38:34.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:34 smithi096 ceph-mon[31930]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 -- 172.21.15.96:0/1076934412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c06b420 msgr2=0x7fa17c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 --2- 172.21.15.96:0/1076934412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c06b420 0x7fa17c106f20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa1700099c0 tx=0x7fa170009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 -- 172.21.15.96:0/1076934412 shutdown_connections 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 --2- 172.21.15.96:0/1076934412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa17c107650 0x7fa17c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 --2- 172.21.15.96:0/1076934412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06aa70 0x7fa17c06ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 --2- 172.21.15.96:0/1076934412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c06b420 0x7fa17c106f20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 -- 172.21.15.96:0/1076934412 >> 172.21.15.96:0/1076934412 conn(0x7fa17c077220 msgr2=0x7fa17c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 -- 172.21.15.96:0/1076934412 shutdown_connections 2024-03-18T15:38:34.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.756+0000 7fa182914700 1 -- 172.21.15.96:0/1076934412 wait complete. 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 Processor -- start 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 -- start start 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa17c06aa70 0x7fa17c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06b420 0x7fa17c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 0x7fa17c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c1125c0 con 0x7fa17c107650 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c112440 con 0x7fa17c06b420 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa182914700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa17c112740 con 0x7fa17c06aa70 2024-03-18T15:38:34.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 0x7fa17c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 0x7fa17c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40344/0 (socket says 172.21.15.96:40344) 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 -- 172.21.15.96:0/4044174749 learned_addr learned my addr 172.21.15.96:0/4044174749 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa17bfff700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa17c06aa70 0x7fa17c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa17b7fe700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06b420 0x7fa17c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 -- 172.21.15.96:0/4044174749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa17c06aa70 msgr2=0x7fa17c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa17c06aa70 0x7fa17c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 -- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06b420 msgr2=0x7fa17c19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06b420 0x7fa17c19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa180eb1700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17c199020 con 0x7fa17c107650 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.757+0000 7fa17b7fe700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa17c06b420 0x7fa17c19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa180eb1700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 0x7fa17c1988c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa17400ef30 tx=0x7fa17400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1740092e0 con 0x7fa17c107650 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17c1992b0 con 0x7fa17c107650 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa17c1a55d0 con 0x7fa17c107650 2024-03-18T15:38:34.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa174004bb0 con 0x7fa17c107650 2024-03-18T15:38:34.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.758+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174004280 con 0x7fa17c107650 2024-03-18T15:38:34.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.759+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa17c062a20 con 0x7fa17c107650 2024-03-18T15:38:34.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.760+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fa17401b030 con 0x7fa17c107650 2024-03-18T15:38:34.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.760+0000 7fa1797fa700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1640a29a0 0x7fa1640a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.760+0000 7fa17bfff700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1640a29a0 0x7fa1640a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.760+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa1740ced40 con 0x7fa17c107650 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa1797fa700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] conn(0x7fa1640ac180 0x7fa1640ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa17401fe50 con 0x7fa1640ac180 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa1740cf220 con 0x7fa17c107650 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa17bfff700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1640a29a0 0x7fa1640a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa17c199a00 tx=0x7fa16c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:34.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa17b7fe700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] conn(0x7fa1640ac180 0x7fa1640ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:34.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.761+0000 7fa17b7fe700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] conn(0x7fa1640ac180 0x7fa1640ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:34.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.762+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== osd.4 v2:172.21.15.137:6808/1772517973 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa17401fe50 con 0x7fa1640ac180 2024-03-18T15:38:34.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.778+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 --> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa17c062e20 con 0x7fa1640ac180 2024-03-18T15:38:34.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.778+0000 7fa1797fa700 1 -- 172.21.15.96:0/4044174749 <== osd.4 v2:172.21.15.137:6808/1772517973 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa17c062e20 con 0x7fa1640ac180 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.778+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] conn(0x7fa1640ac180 msgr2=0x7fa1640ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.778+0000 7fa182914700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] conn(0x7fa1640ac180 0x7fa1640ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1640a29a0 msgr2=0x7fa1640a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1640a29a0 0x7fa1640a4e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa17c199a00 tx=0x7fa16c009260 comp rx=0 tx=0).stop 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 msgr2=0x7fa17c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 --2- 172.21.15.96:0/4044174749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa17c107650 0x7fa17c1988c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa17400ef30 tx=0x7fa17400c3a0 comp rx=0 tx=0).stop 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa180eb1700 1 -- 172.21.15.96:0/4044174749 reap_dead start 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 shutdown_connections 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 >> 172.21.15.96:0/4044174749 conn(0x7fa17c077220 msgr2=0x7fa17c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 shutdown_connections 2024-03-18T15:38:34.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:34.779+0000 7fa182914700 1 -- 172.21.15.96:0/4044174749 wait complete. 2024-03-18T15:38:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:34 smithi137 ceph-mon[35868]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T15:38:34.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:34 smithi193 ceph-mon[36702]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-18T15:38:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:36 smithi137 ceph-mon[35868]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T15:38:36.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:36 smithi193 ceph-mon[36702]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T15:38:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:36 smithi096 ceph-mon[31930]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 -- 172.21.15.96:0/2495026106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 msgr2=0x7f7d68101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 --2- 172.21.15.96:0/2495026106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d68101630 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098f0 tx=0x7f7d5c009c00 comp rx=0 tx=0).stop 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 -- 172.21.15.96:0/2495026106 shutdown_connections 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 --2- 172.21.15.96:0/2495026106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68101b70 0x7f7d6810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 --2- 172.21.15.96:0/2495026106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68107260 0x7f7d68107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 --2- 172.21.15.96:0/2495026106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d68101630 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 -- 172.21.15.96:0/2495026106 >> 172.21.15.96:0/2495026106 conn(0x7f7d680782a0 msgr2=0x7f7d680786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 -- 172.21.15.96:0/2495026106 shutdown_connections 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.185+0000 7f7d6d396700 1 -- 172.21.15.96:0/2495026106 wait complete. 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 Processor -- start 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 -- start start 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d6819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68101b70 0x7f7d6819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68107260 0x7f7d68198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d68111470 con 0x7f7d681011b0 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d681112f0 con 0x7f7d68107260 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d6d396700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d681115f0 con 0x7f7d68101b70 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d6819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d66ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d6819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40368/0 (socket says 172.21.15.96:40368) 2024-03-18T15:38:37.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d66ffd700 1 -- 172.21.15.96:0/357685692 learned_addr learned my addr 172.21.15.96:0/357685692 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d667fc700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68101b70 0x7f7d6819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.186+0000 7f7d677fe700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68107260 0x7f7d68198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 -- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68101b70 msgr2=0x7f7d6819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68101b70 0x7f7d6819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 -- 172.21.15.96:0/357685692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68107260 msgr2=0x7f7d68198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68107260 0x7f7d68198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d68199030 con 0x7f7d681011b0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d667fc700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d68101b70 0x7f7d6819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d677fe700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d68107260 0x7f7d68198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d66ffd700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d6819e680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ec10 tx=0x7f7d5800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cc90 con 0x7f7d681011b0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d68199320 con 0x7f7d681011b0 2024-03-18T15:38:37.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5800ce30 con 0x7f7d681011b0 2024-03-18T15:38:37.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58010690 con 0x7f7d681011b0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.187+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d681a54e0 con 0x7f7d681011b0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.188+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d6806ad40 con 0x7f7d681011b0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.189+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7d58003680 con 0x7f7d681011b0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d4ffff700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d500a2920 0x7f7d500a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d667fc700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d500a2920 0x7f7d500a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7d580cf030 con 0x7f7d681011b0 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d4ffff700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] conn(0x7f7d500ac160 0x7f7d500ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d58012ec0 con 0x7f7d500ac160 2024-03-18T15:38:37.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7d580cf510 con 0x7f7d681011b0 2024-03-18T15:38:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d667fc700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d500a2920 0x7f7d500a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098c0 tx=0x7f7d5c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.190+0000 7f7d677fe700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] conn(0x7f7d500ac160 0x7f7d500ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.191+0000 7f7d677fe700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] conn(0x7f7d500ac160 0x7f7d500ae580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.192+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== osd.8 v2:172.21.15.193:6816/1156737532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d58012ec0 con 0x7f7d500ac160 2024-03-18T15:38:37.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.208+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 --> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d681028a0 con 0x7f7d500ac160 2024-03-18T15:38:37.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.208+0000 7f7d4ffff700 1 -- 172.21.15.96:0/357685692 <== osd.8 v2:172.21.15.193:6816/1156737532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d681028a0 con 0x7f7d500ac160 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] conn(0x7f7d500ac160 msgr2=0x7f7d500ae580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] conn(0x7f7d500ac160 0x7f7d500ae580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d500a2920 msgr2=0x7f7d500a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d500a2920 0x7f7d500a4de0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0098c0 tx=0x7f7d5c019040 comp rx=0 tx=0).stop 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 msgr2=0x7f7d6819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 --2- 172.21.15.96:0/357685692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d681011b0 0x7f7d6819e680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ec10 tx=0x7f7d5800ef20 comp rx=0 tx=0).stop 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d677fe700 1 -- 172.21.15.96:0/357685692 reap_dead start 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 shutdown_connections 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 >> 172.21.15.96:0/357685692 conn(0x7f7d680782a0 msgr2=0x7f7d68100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 shutdown_connections 2024-03-18T15:38:37.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:37.209+0000 7f7d6d396700 1 -- 172.21.15.96:0/357685692 wait complete. 2024-03-18T15:38:38.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:38 smithi096 ceph-mon[31930]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-18T15:38:38.740 INFO:teuthology.orchestra.run.smithi096.stdout:141733920824 2024-03-18T15:38:38.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-18T15:38:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:38 smithi137 ceph-mon[35868]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-18T15:38:38.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:38 smithi193 ceph-mon[36702]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 -- 172.21.15.96:0/2292432880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 msgr2=0x7f5e2c101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2292432880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c101930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009990 tx=0x7f5e20009ca0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 -- 172.21.15.96:0/2292432880 shutdown_connections 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2292432880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c101e70 0x7f5e2c10e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2292432880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e2c1061d0 0x7f5e2c1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2292432880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c101930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 -- 172.21.15.96:0/2292432880 >> 172.21.15.96:0/2292432880 conn(0x7f5e2c078090 msgr2=0x7f5e2c0784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 -- 172.21.15.96:0/2292432880 shutdown_connections 2024-03-18T15:38:38.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.937+0000 7f5e3088d700 1 -- 172.21.15.96:0/2292432880 wait complete. 2024-03-18T15:38:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.938+0000 7f5e3088d700 1 Processor -- start 2024-03-18T15:38:38.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.938+0000 7f5e3088d700 1 -- start start 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e2c101e70 0x7f5e2c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c1061d0 0x7f5e2c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c111540 con 0x7f5e2c1014b0 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1113c0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e3088d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1116c0 con 0x7f5e2c101e70 2024-03-18T15:38:38.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40396/0 (socket says 172.21.15.96:40396) 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2a59c700 1 -- 172.21.15.96:0/2988604251 learned_addr learned my addr 172.21.15.96:0/2988604251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c1061d0 0x7f5e2c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e29d9b700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e2c101e70 0x7f5e2c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 -- 172.21.15.96:0/2988604251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e2c101e70 msgr2=0x7f5e2c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e2c101e70 0x7f5e2c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 -- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 msgr2=0x7f5e2c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1a6f80 con 0x7f5e2c1061d0 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2a59c700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2c1014b0 0x7f5e2c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.939+0000 7f5e2ad9d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c1061d0 0x7f5e2c198190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400efc0 tx=0x7f5e2400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:38.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e240092e0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e24004bb0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24004280 con 0x7f5e2c1061d0 2024-03-18T15:38:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1a7210 con 0x7f5e2c1061d0 2024-03-18T15:38:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1a76d0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.940+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e2c0629e0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f5e2401b030 con 0x7f5e2c1061d0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e140a29a0 0x7f5e140a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5e240ceec0 con 0x7f5e2c1061d0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e2a59c700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e140a29a0 0x7f5e140a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] conn(0x7f5e140ac180 0x7f5e140ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e2401fe50 con 0x7f5e140ac180 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.942+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5e240cf270 con 0x7f5e2c1061d0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.943+0000 7f5e29d9b700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] conn(0x7f5e140ac180 0x7f5e140ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.943+0000 7f5e2a59c700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e140a29a0 0x7f5e140a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c011550 tx=0x7f5e1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:38.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.943+0000 7f5e29d9b700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] conn(0x7f5e140ac180 0x7f5e140ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:38.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.944+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== osd.7 v2:172.21.15.137:6816/1840494271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e2401fe50 con 0x7f5e140ac180 2024-03-18T15:38:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.960+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 --> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e2c062e60 con 0x7f5e140ac180 2024-03-18T15:38:38.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.960+0000 7f5e1b7fe700 1 -- 172.21.15.96:0/2988604251 <== osd.7 v2:172.21.15.137:6816/1840494271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5e2c062e60 con 0x7f5e140ac180 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] conn(0x7f5e140ac180 msgr2=0x7f5e140ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] conn(0x7f5e140ac180 0x7f5e140ae5a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e140a29a0 msgr2=0x7f5e140a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e140a29a0 0x7f5e140a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c011550 tx=0x7f5e1c009260 comp rx=0 tx=0).stop 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c1061d0 msgr2=0x7f5e2c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 --2- 172.21.15.96:0/2988604251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e2c1061d0 0x7f5e2c198190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400efc0 tx=0x7f5e2400c3a0 comp rx=0 tx=0).stop 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e2ad9d700 1 -- 172.21.15.96:0/2988604251 reap_dead start 2024-03-18T15:38:38.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 shutdown_connections 2024-03-18T15:38:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 >> 172.21.15.96:0/2988604251 conn(0x7f5e2c078090 msgr2=0x7f5e2c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 shutdown_connections 2024-03-18T15:38:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:38.961+0000 7f5e3088d700 1 -- 172.21.15.96:0/2988604251 wait complete. 2024-03-18T15:38:40.854 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:40 smithi096 ceph-mon[31930]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T15:38:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:40 smithi137 ceph-mon[35868]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T15:38:40.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:40 smithi193 ceph-mon[36702]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.681+0000 7fc3602bb700 1 -- 172.21.15.96:0/2128618696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 msgr2=0x7fc3581028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.681+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2128618696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3581028f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3540099c0 tx=0x7fc354009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- 172.21.15.96:0/2128618696 shutdown_connections 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2128618696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3581075f0 0x7fc35810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2128618696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3581028f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2128618696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc358102ec0 0x7fc358106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- 172.21.15.96:0/2128618696 >> 172.21.15.96:0/2128618696 conn(0x7fc3580fdd80 msgr2=0x7fc3581001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- 172.21.15.96:0/2128618696 shutdown_connections 2024-03-18T15:38:41.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- 172.21.15.96:0/2128618696 wait complete. 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 Processor -- start 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- start start 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3580745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc358102ec0 0x7fc358074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3581075f0 0x7fc358075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc3581125d0 con 0x7fc358102510 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc358112450 con 0x7fc3581075f0 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.682+0000 7fc3602bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc358112750 con 0x7fc358102ec0 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3580745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3580745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40420/0 (socket says 172.21.15.96:40420) 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 -- 172.21.15.96:0/2909057631 learned_addr learned my addr 172.21.15.96:0/2909057631 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e858700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3581075f0 0x7fc358075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35d856700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc358102ec0 0x7fc358074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:41.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 -- 172.21.15.96:0/2909057631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc358102ec0 msgr2=0x7fc358074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc358102ec0 0x7fc358074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 -- 172.21.15.96:0/2909057631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3581075f0 msgr2=0x7fc358075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3581075f0 0x7fc358075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3581a6fb0 con 0x7fc358102510 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc35e057700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3580745d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc354009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354026070 con 0x7fc358102510 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3581a7240 con 0x7fc358102510 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc354004cd0 con 0x7fc358102510 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354003ba0 con 0x7fc358102510 2024-03-18T15:38:41.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.683+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3581a76f0 con 0x7fc358102510 2024-03-18T15:38:41.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.684+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3580629e0 con 0x7fc358102510 2024-03-18T15:38:41.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.685+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc354029070 con 0x7fc358102510 2024-03-18T15:38:41.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc34f7fe700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3440a29a0 0x7fc3440a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc35d856700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3440a29a0 0x7fc3440a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc3540cfa00 con 0x7fc358102510 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc34f7fe700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] conn(0x7fc3440ac180 0x7fc3440ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3440aec50 con 0x7fc3440ac180 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc3540cfee0 con 0x7fc358102510 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc35e858700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] conn(0x7fc3440ac180 0x7fc3440ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc35d856700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3440a29a0 0x7fc3440a4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc358108be0 tx=0x7fc3480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.686+0000 7fc35e858700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] conn(0x7fc3440ac180 0x7fc3440ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:41.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.687+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== osd.0 v2:172.21.15.96:6802/3832630015 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3440aec50 con 0x7fc3440ac180 2024-03-18T15:38:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.703+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 --> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc358062e60 con 0x7fc3440ac180 2024-03-18T15:38:41.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.703+0000 7fc34f7fe700 1 -- 172.21.15.96:0/2909057631 <== osd.0 v2:172.21.15.96:6802/3832630015 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc358062e60 con 0x7fc3440ac180 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] conn(0x7fc3440ac180 msgr2=0x7fc3440ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] conn(0x7fc3440ac180 0x7fc3440ae5a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3440a29a0 msgr2=0x7fc3440a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc3440a29a0 0x7fc3440a4e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc358108be0 tx=0x7fc3480073d0 comp rx=0 tx=0).stop 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 msgr2=0x7fc3580745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 --2- 172.21.15.96:0/2909057631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc358102510 0x7fc3580745d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc354009f80 comp rx=0 tx=0).stop 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc35e858700 1 -- 172.21.15.96:0/2909057631 reap_dead start 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 shutdown_connections 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 >> 172.21.15.96:0/2909057631 conn(0x7fc3580fdd80 msgr2=0x7fc358100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 shutdown_connections 2024-03-18T15:38:41.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:41.704+0000 7fc3602bb700 1 -- 172.21.15.96:0/2909057631 wait complete. 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- 172.21.15.96:0/3520267381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102510 msgr2=0x7f4e141028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- 172.21.15.96:0/3520267381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102510 0x7f4e141028f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b0d0 tx=0x7f4e1000b3e0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- 172.21.15.96:0/3520267381 shutdown_connections 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- 172.21.15.96:0/3520267381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e141075f0 0x7f4e1410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- 172.21.15.96:0/3520267381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102510 0x7f4e141028f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- 172.21.15.96:0/3520267381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e14102ec0 0x7f4e14106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- 172.21.15.96:0/3520267381 >> 172.21.15.96:0/3520267381 conn(0x7f4e140fdd80 msgr2=0x7f4e141001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- 172.21.15.96:0/3520267381 shutdown_connections 2024-03-18T15:38:42.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- 172.21.15.96:0/3520267381 wait complete. 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 Processor -- start 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- start start 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e14102510 0x7f4e1419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 0x7f4e1419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e141075f0 0x7f4e14198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141125d0 con 0x7f4e14102ec0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14112450 con 0x7f4e14102510 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e1b763700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14112750 con 0x7f4e141075f0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 0x7f4e1419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 0x7f4e1419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40436/0 (socket says 172.21.15.96:40436) 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 -- 172.21.15.96:0/2934224643 learned_addr learned my addr 172.21.15.96:0/2934224643 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e194ff700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e14102510 0x7f4e1419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e19d00700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e141075f0 0x7f4e14198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 -- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e141075f0 msgr2=0x7f4e14198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e141075f0 0x7f4e14198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 -- 172.21.15.96:0/2934224643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e14102510 msgr2=0x7f4e1419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e14102510 0x7f4e1419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e14198fd0 con 0x7f4e14102ec0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e18cfe700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 0x7f4e1419ebf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e970 tx=0x7f4e0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.161+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04004a30 con 0x7f4e14102ec0 2024-03-18T15:38:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.162+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e0400ccc0 con 0x7f4e14102ec0 2024-03-18T15:38:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.162+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e141992c0 con 0x7f4e14102ec0 2024-03-18T15:38:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.162+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e04005650 con 0x7f4e14102ec0 2024-03-18T15:38:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.162+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a5580 con 0x7f4e14102ec0 2024-03-18T15:38:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.162+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e14062a60 con 0x7f4e14102ec0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4e04004bd0 con 0x7f4e14102ec0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4e000a29a0 0x7f4e000a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e194ff700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4e000a29a0 0x7f4e000a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4e040cf040 con 0x7f4e14102ec0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] conn(0x7f4e000ac180 0x7f4e000ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e04015e80 con 0x7f4e000ac180 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4e040cf520 con 0x7f4e14102ec0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e194ff700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4e000a29a0 0x7f4e000a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009fd0 tx=0x7f4e1000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e19d00700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] conn(0x7f4e000ac180 0x7f4e000ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.165+0000 7f4e19d00700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] conn(0x7f4e000ac180 0x7f4e000ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.167+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== osd.2 v2:172.21.15.193:6800/110382863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e04015e80 con 0x7f4e000ac180 2024-03-18T15:38:42.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.182+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 --> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e14062e60 con 0x7f4e000ac180 2024-03-18T15:38:42.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.182+0000 7f4e0a7fc700 1 -- 172.21.15.96:0/2934224643 <== osd.2 v2:172.21.15.193:6800/110382863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4e14062e60 con 0x7f4e000ac180 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] conn(0x7f4e000ac180 msgr2=0x7f4e000ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] conn(0x7f4e000ac180 0x7f4e000ae5a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4e000a29a0 msgr2=0x7f4e000a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4e000a29a0 0x7f4e000a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009fd0 tx=0x7f4e1000b040 comp rx=0 tx=0).stop 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 msgr2=0x7f4e1419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 --2- 172.21.15.96:0/2934224643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e14102ec0 0x7f4e1419ebf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400e970 tx=0x7f4e0400ec80 comp rx=0 tx=0).stop 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e19d00700 1 -- 172.21.15.96:0/2934224643 reap_dead start 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 shutdown_connections 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 >> 172.21.15.96:0/2934224643 conn(0x7f4e140fdd80 msgr2=0x7f4e140ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:42.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 shutdown_connections 2024-03-18T15:38:42.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.183+0000 7f4e1b763700 1 -- 172.21.15.96:0/2934224643 wait complete. 2024-03-18T15:38:42.617 INFO:teuthology.orchestra.run.smithi096.stdout:257698037796 2024-03-18T15:38:42.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-18T15:38:42.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 -- 172.21.15.96:0/3286382790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 msgr2=0x7f5280106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 --2- 172.21.15.96:0/3286382790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f5280106f60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f52740099c0 tx=0x7f5274009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.805 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:42 smithi096 ceph-mon[31930]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 -- 172.21.15.96:0/3286382790 shutdown_connections 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 --2- 172.21.15.96:0/3286382790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280107690 0x7f528010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 --2- 172.21.15.96:0/3286382790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f5280101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 --2- 172.21.15.96:0/3286382790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f5280106f60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 -- 172.21.15.96:0/3286382790 >> 172.21.15.96:0/3286382790 conn(0x7f5280076510 msgr2=0x7f5280076920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 -- 172.21.15.96:0/3286382790 shutdown_connections 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.805+0000 7f5284b1d700 1 -- 172.21.15.96:0/3286382790 wait complete. 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 Processor -- start 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 -- start start 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f528019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f528019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280107690 0x7f52801988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112600 con 0x7f5280101820 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112480 con 0x7f5280107690 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f5284b1d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112780 con 0x7f5280100e70 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f528019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f528019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f528019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:37928/0 (socket says 172.21.15.96:37928) 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 -- 172.21.15.96:0/2677391246 learned_addr learned my addr 172.21.15.96:0/2677391246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527ed9d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280107690 0x7f52801988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 -- 172.21.15.96:0/2677391246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280107690 msgr2=0x7f52801988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5280107690 0x7f52801988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 -- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 msgr2=0x7f528019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.806+0000 7f527e59c700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f528019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f527e59c700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280199040 con 0x7f5280100e70 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f527dd9b700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5280101820 0x7f528019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:42.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f527e59c700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f528019e720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f527000e9c0 tx=0x7f527000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527000cbd0 con 0x7f5280100e70 2024-03-18T15:38:42.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5280199330 con 0x7f5280100e70 2024-03-18T15:38:42.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527000ee60 con 0x7f5280100e70 2024-03-18T15:38:42.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270005630 con 0x7f5280100e70 2024-03-18T15:38:42.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.807+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a55f0 con 0x7f5280100e70 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.808+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5280062a60 con 0x7f5280100e70 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.809+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f5270004750 con 0x7f5280100e70 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f526f7fe700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f52680a2920 0x7f52680a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f527dd9b700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f52680a2920 0x7f52680a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f52700cf0e0 con 0x7f5280100e70 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f526f7fe700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] conn(0x7f52680ac100 0x7f52680ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f527001fe50 con 0x7f52680ac100 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f52700cf5c0 con 0x7f5280100e70 2024-03-18T15:38:42.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f527dd9b700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f52680a2920 0x7f52680a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5274009990 tx=0x7f5274005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.810+0000 7f527ed9d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] conn(0x7f52680ac100 0x7f52680ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:42.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.811+0000 7f527ed9d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] conn(0x7f52680ac100 0x7f52680ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:42.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.811+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== osd.5 v2:172.21.15.193:6808/298847477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f527001fe50 con 0x7f52680ac100 2024-03-18T15:38:42.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.827+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 --> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5280062ee0 con 0x7f52680ac100 2024-03-18T15:38:42.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.827+0000 7f526f7fe700 1 -- 172.21.15.96:0/2677391246 <== osd.5 v2:172.21.15.193:6808/298847477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5280062ee0 con 0x7f52680ac100 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] conn(0x7f52680ac100 msgr2=0x7f52680ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] conn(0x7f52680ac100 0x7f52680ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f52680a2920 msgr2=0x7f52680a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f52680a2920 0x7f52680a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5274009990 tx=0x7f5274005fb0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 msgr2=0x7f528019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 --2- 172.21.15.96:0/2677391246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280100e70 0x7f528019e720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f527000e9c0 tx=0x7f527000ecd0 comp rx=0 tx=0).stop 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f527ed9d700 1 -- 172.21.15.96:0/2677391246 reap_dead start 2024-03-18T15:38:42.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 shutdown_connections 2024-03-18T15:38:42.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 >> 172.21.15.96:0/2677391246 conn(0x7f5280076510 msgr2=0x7f52800fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:42.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 shutdown_connections 2024-03-18T15:38:42.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:42.828+0000 7f5284b1d700 1 -- 172.21.15.96:0/2677391246 wait complete. 2024-03-18T15:38:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:42 smithi137 ceph-mon[35868]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-18T15:38:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:42 smithi193 ceph-mon[36702]: pgmap v306: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-18T15:38:43.705 INFO:teuthology.orchestra.run.smithi096.stdout:223338299434 2024-03-18T15:38:43.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-18T15:38:44.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.296+0000 7ff45887d700 1 -- 172.21.15.96:0/1904827012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4541064e0 msgr2=0x7ff45410e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.296+0000 7ff45887d700 1 --2- 172.21.15.96:0/1904827012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4541064e0 0x7ff45410e0b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00b0a0 tx=0x7ff44c00b3b0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 -- 172.21.15.96:0/1904827012 shutdown_connections 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 --2- 172.21.15.96:0/1904827012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4541064e0 0x7ff45410e0b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 --2- 172.21.15.96:0/1904827012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101460 0x7ff454101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 --2- 172.21.15.96:0/1904827012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff454101e10 0x7ff454105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 -- 172.21.15.96:0/1904827012 >> 172.21.15.96:0/1904827012 conn(0x7ff454077210 msgr2=0x7ff454077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 -- 172.21.15.96:0/1904827012 shutdown_connections 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 -- 172.21.15.96:0/1904827012 wait complete. 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 Processor -- start 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.297+0000 7ff45887d700 1 -- start start 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101460 0x7ff454199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 0x7ff454199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4541064e0 0x7ff4541a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4541114c0 con 0x7ff454101e10 2024-03-18T15:38:44.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111340 con 0x7ff4541064e0 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff454111640 con 0x7ff454101460 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 0x7ff454199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 0x7ff454199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44006/0 (socket says 172.21.15.96:44006) 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 -- 172.21.15.96:0/1577785206 learned_addr learned my addr 172.21.15.96:0/1577785206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45259c700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101460 0x7ff454199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff452d9d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4541064e0 0x7ff4541a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 -- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101460 msgr2=0x7ff454199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff454101460 0x7ff454199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 -- 172.21.15.96:0/1577785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4541064e0 msgr2=0x7ff4541a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff4541064e0 0x7ff4541a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4541a4e00 con 0x7ff454101e10 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff451d9b700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 0x7ff454199d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff44800e970 tx=0x7ff44800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448004a30 con 0x7ff454101e10 2024-03-18T15:38:44.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff44800ccc0 con 0x7ff454101e10 2024-03-18T15:38:44.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4541a5090 con 0x7ff454101e10 2024-03-18T15:38:44.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448005650 con 0x7ff454101e10 2024-03-18T15:38:44.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.298+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4541a55a0 con 0x7ff454101e10 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.299+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4540629e0 con 0x7ff454101e10 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.300+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7ff448004bd0 con 0x7ff454101e10 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff4437fe700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff43c0a2920 0x7ff43c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff45259c700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff43c0a2920 0x7ff43c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff4480cebb0 con 0x7ff454101e10 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff4437fe700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] conn(0x7ff43c0ac100 0x7ff43c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff44800fe50 con 0x7ff43c0ac100 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff4480d1300 con 0x7ff454101e10 2024-03-18T15:38:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff452d9d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] conn(0x7ff43c0ac100 0x7ff43c0ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:44.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.301+0000 7ff45259c700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff43c0a2920 0x7ff43c0a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff444009c60 tx=0x7ff444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:44.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.302+0000 7ff452d9d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] conn(0x7ff43c0ac100 0x7ff43c0ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:44.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.303+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== osd.11 v2:172.21.15.193:6824/97980715 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff44800fe50 con 0x7ff43c0ac100 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.320+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 --> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff454062e60 con 0x7ff43c0ac100 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff4437fe700 1 -- 172.21.15.96:0/1577785206 <== osd.11 v2:172.21.15.193:6824/97980715 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff454062e60 con 0x7ff43c0ac100 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] conn(0x7ff43c0ac100 msgr2=0x7ff43c0ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] conn(0x7ff43c0ac100 0x7ff43c0ae520 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff43c0a2920 msgr2=0x7ff43c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff43c0a2920 0x7ff43c0a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff444009c60 tx=0x7ff444009260 comp rx=0 tx=0).stop 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 msgr2=0x7ff454199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:44.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 --2- 172.21.15.96:0/1577785206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff454101e10 0x7ff454199d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff44800e970 tx=0x7ff44800ec80 comp rx=0 tx=0).stop 2024-03-18T15:38:44.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff452d9d700 1 -- 172.21.15.96:0/1577785206 reap_dead start 2024-03-18T15:38:44.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 shutdown_connections 2024-03-18T15:38:44.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.321+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 >> 172.21.15.96:0/1577785206 conn(0x7ff454077210 msgr2=0x7ff4540ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:44.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.322+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 shutdown_connections 2024-03-18T15:38:44.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:44.322+0000 7ff45887d700 1 -- 172.21.15.96:0/1577785206 wait complete. 2024-03-18T15:38:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:44 smithi137 ceph-mon[35868]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:44.884 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:44 smithi096 ceph-mon[31930]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:44.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:44 smithi193 ceph-mon[36702]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:45.541 INFO:teuthology.orchestra.run.smithi096.stdout:34359738445 2024-03-18T15:38:45.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-18T15:38:46.365 INFO:teuthology.orchestra.run.smithi096.stdout:167503724597 2024-03-18T15:38:46.365 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-18T15:38:46.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 -- 172.21.15.96:0/253452888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48081064b0 msgr2=0x7f480810e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 --2- 172.21.15.96:0/253452888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48081064b0 0x7f480810e080 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f480000b0d0 tx=0x7f480000b3e0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 -- 172.21.15.96:0/253452888 shutdown_connections 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 --2- 172.21.15.96:0/253452888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48081064b0 0x7f480810e080 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 --2- 172.21.15.96:0/253452888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 0x7f4808101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 --2- 172.21.15.96:0/253452888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4808101e10 0x7f4808105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 -- 172.21.15.96:0/253452888 >> 172.21.15.96:0/253452888 conn(0x7f4808077220 msgr2=0x7f4808077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 -- 172.21.15.96:0/253452888 shutdown_connections 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.538+0000 7f480faf1700 1 -- 172.21.15.96:0/253452888 wait complete. 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 Processor -- start 2024-03-18T15:38:46.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 -- start start 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 0x7f48081998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 0x7f4808199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48081064b0 0x7f48081a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48081114f0 con 0x7f4808101e10 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111370 con 0x7f48081064b0 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480faf1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4808111670 con 0x7f4808101460 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480d08c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 0x7f4808199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480d08c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 0x7f4808199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44026/0 (socket says 172.21.15.96:44026) 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480d88d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 0x7f48081998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:46.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480d08c700 1 -- 172.21.15.96:0/3051850641 learned_addr learned my addr 172.21.15.96:0/3051850641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.539+0000 7f480e08e700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48081064b0 0x7f48081a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 -- 172.21.15.96:0/3051850641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 msgr2=0x7f48081998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 0x7f48081998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 -- 172.21.15.96:0/3051850641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48081064b0 msgr2=0x7f48081a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48081064b0 0x7f48081a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48081a4e70 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d88d700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4808101460 0x7f48081998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480d08c700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 0x7f4808199df0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d950 tx=0x7f47f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8016070 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4808197700 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f8004d10 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f8005600 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.540+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4808197c90 con 0x7f4808101e10 2024-03-18T15:38:46.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.541+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f48080629e0 con 0x7f4808101e10 2024-03-18T15:38:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.542+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f47f8004220 con 0x7f4808101e10 2024-03-18T15:38:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f47feffd700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47f40a2920 0x7f47f40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f480d88d700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47f40a2920 0x7f47f40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f47f80ced30 con 0x7f4808101e10 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f47feffd700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] conn(0x7f47f40ac100 0x7f47f40ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47f8015ec0 con 0x7f47f40ac100 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f47f80cf210 con 0x7f4808101e10 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f480e08e700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] conn(0x7f47f40ac100 0x7f47f40ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.543+0000 7f480d88d700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47f40a2920 0x7f47f40a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4804011490 tx=0x7f4804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.544+0000 7f480e08e700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] conn(0x7f47f40ac100 0x7f47f40ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.545+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== osd.9 v2:172.21.15.96:6826/205742436 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47f8015ec0 con 0x7f47f40ac100 2024-03-18T15:38:46.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.565+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 --> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4808062de0 con 0x7f47f40ac100 2024-03-18T15:38:46.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f47feffd700 1 -- 172.21.15.96:0/3051850641 <== osd.9 v2:172.21.15.96:6826/205742436 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4808062de0 con 0x7f47f40ac100 2024-03-18T15:38:46.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] conn(0x7f47f40ac100 msgr2=0x7f47f40ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] conn(0x7f47f40ac100 0x7f47f40ae520 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:46.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47f40a2920 msgr2=0x7f47f40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47f40a2920 0x7f47f40a4de0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4804011490 tx=0x7f4804009260 comp rx=0 tx=0).stop 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 msgr2=0x7f4808199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 --2- 172.21.15.96:0/3051850641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4808101e10 0x7f4808199df0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f47f800d950 tx=0x7f47f800dc60 comp rx=0 tx=0).stop 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480e08e700 1 -- 172.21.15.96:0/3051850641 reap_dead start 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 shutdown_connections 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.566+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 >> 172.21.15.96:0/3051850641 conn(0x7f4808077220 msgr2=0x7f48080ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.567+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 shutdown_connections 2024-03-18T15:38:46.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:46.567+0000 7f480faf1700 1 -- 172.21.15.96:0/3051850641 wait complete. 2024-03-18T15:38:46.649 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:46 smithi096 ceph-mon[31930]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:46 smithi137 ceph-mon[35868]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:46.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:46 smithi193 ceph-mon[36702]: pgmap v308: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 -- 172.21.15.96:0/3944617462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f479806b420 msgr2=0x7f4798106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 --2- 172.21.15.96:0/3944617462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f479806b420 0x7f4798106f20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f478c0099c0 tx=0x7f478c009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 -- 172.21.15.96:0/3944617462 shutdown_connections 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 --2- 172.21.15.96:0/3944617462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4798107650 0x7f479810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 --2- 172.21.15.96:0/3944617462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479806ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 --2- 172.21.15.96:0/3944617462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f479806b420 0x7f4798106f20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 -- 172.21.15.96:0/3944617462 >> 172.21.15.96:0/3944617462 conn(0x7f4798077220 msgr2=0x7f4798077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 -- 172.21.15.96:0/3944617462 shutdown_connections 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.003+0000 7f479d534700 1 -- 172.21.15.96:0/3944617462 wait complete. 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 Processor -- start 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 -- start start 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479806b420 0x7f479819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 0x7f4798198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47981125c0 con 0x7f4798107650 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112440 con 0x7f479806b420 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f479d534700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4798112740 con 0x7f479806aa70 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f47977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 0x7f4798198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f4796ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f47977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 0x7f4798198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44038/0 (socket says 172.21.15.96:44038) 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f47977fe700 1 -- 172.21.15.96:0/570847682 learned_addr learned my addr 172.21.15.96:0/570847682 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f4796ffd700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33666/0 (socket says 172.21.15.96:33666) 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f47967fc700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479806b420 0x7f479819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f4796ffd700 1 -- 172.21.15.96:0/570847682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479806b420 msgr2=0x7f479819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.004+0000 7f4796ffd700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f479806b420 0x7f479819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f4796ffd700 1 -- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 msgr2=0x7f4798198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f4796ffd700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 0x7f4798198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f4796ffd700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47981990c0 con 0x7f479806aa70 2024-03-18T15:38:47.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f47977fe700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4798107650 0x7f4798198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f4796ffd700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479819e7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f478800e9f0 tx=0x7f478800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788004220 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788004d10 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47981993b0 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788005020 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.005+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981a5670 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.006+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47980629e0 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.007+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f4788004750 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.007+0000 7f477ffff700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47800a2920 0x7f47800a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f47967fc700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47800a2920 0x7f47800a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f47880ceb60 con 0x7f479806aa70 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f477ffff700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] conn(0x7f47800ac100 0x7f47800ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f478800ae50 con 0x7f47800ac100 2024-03-18T15:38:47.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f47880cef10 con 0x7f479806aa70 2024-03-18T15:38:47.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f47977fe700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] conn(0x7f47800ac100 0x7f47800ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f47967fc700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47800a2920 0x7f47800a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f478c009990 tx=0x7f478c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.008+0000 7f47977fe700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] conn(0x7f47800ac100 0x7f47800ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.009+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== osd.3 v2:172.21.15.96:6810/157229437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f478800ae50 con 0x7f47800ac100 2024-03-18T15:38:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.025+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 --> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4798062e60 con 0x7f47800ac100 2024-03-18T15:38:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.025+0000 7f477ffff700 1 -- 172.21.15.96:0/570847682 <== osd.3 v2:172.21.15.96:6810/157229437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4798062e60 con 0x7f47800ac100 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.025+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] conn(0x7f47800ac100 msgr2=0x7f47800ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.025+0000 7f479d534700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] conn(0x7f47800ac100 0x7f47800ae520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47800a2920 msgr2=0x7f47800a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47800a2920 0x7f47800a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f478c009990 tx=0x7f478c005fb0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 msgr2=0x7f479819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 --2- 172.21.15.96:0/570847682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f479806aa70 0x7f479819e7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f478800e9f0 tx=0x7f478800ed00 comp rx=0 tx=0).stop 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f47977fe700 1 -- 172.21.15.96:0/570847682 reap_dead start 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 shutdown_connections 2024-03-18T15:38:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 >> 172.21.15.96:0/570847682 conn(0x7f4798077220 msgr2=0x7f47980ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:47.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 shutdown_connections 2024-03-18T15:38:47.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.026+0000 7f479d534700 1 -- 172.21.15.96:0/570847682 wait complete. 2024-03-18T15:38:47.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.211+0000 7fd476e60700 1 -- 172.21.15.96:0/508172229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470102350 msgr2=0x7fd4701027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.211+0000 7fd476e60700 1 --2- 172.21.15.96:0/508172229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470102350 0x7fd4701027d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0099c0 tx=0x7fd45c009cd0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- 172.21.15.96:0/508172229 shutdown_connections 2024-03-18T15:38:47.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- 172.21.15.96:0/508172229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102d10 0x7fd47010f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- 172.21.15.96:0/508172229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470108360 0x7fd470108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- 172.21.15.96:0/508172229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470102350 0x7fd4701027d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- 172.21.15.96:0/508172229 >> 172.21.15.96:0/508172229 conn(0x7fd470079430 msgr2=0x7fd470079840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- 172.21.15.96:0/508172229 shutdown_connections 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- 172.21.15.96:0/508172229 wait complete. 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 Processor -- start 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- start start 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 0x7fd4700745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470102d10 0x7fd470074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470108360 0x7fd470075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4701125e0 con 0x7fd470108360 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112460 con 0x7fd470102d10 2024-03-18T15:38:47.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.212+0000 7fd476e60700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd470112760 con 0x7fd470102350 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 0x7fd4700745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd4753fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470108360 0x7fd470075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 0x7fd4700745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33680/0 (socket says 172.21.15.96:33680) 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470102d10 0x7fd470074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 -- 172.21.15.96:0/3344270370 learned_addr learned my addr 172.21.15.96:0/3344270370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 -- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470102d10 msgr2=0x7fd470074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470102d10 0x7fd470074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 -- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470108360 msgr2=0x7fd470075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd470108360 0x7fd470075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4701a6f50 con 0x7fd470102350 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd46ffff700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd470102d10 0x7fd470074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd474bfc700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 0x7fd4700745d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9a0 tx=0x7fd46400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46400ca10 con 0x7fd470102350 2024-03-18T15:38:47.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701a7240 con 0x7fd470102350 2024-03-18T15:38:47.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd46400cbb0 con 0x7fd470102350 2024-03-18T15:38:47.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464018680 con 0x7fd470102350 2024-03-18T15:38:47.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.213+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4701a77a0 con 0x7fd470102350 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.214+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4700629e0 con 0x7fd470102350 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.216+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd464010430 con 0x7fd470102350 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.216+0000 7fd46dffb700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4600a2920 0x7fd4600a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46ffff700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4600a2920 0x7fd4600a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd4640cf3f0 con 0x7fd470102350 2024-03-18T15:38:47.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46dffb700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] conn(0x7fd4600ac100 0x7fd4600ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd46401bec0 con 0x7fd4600ac100 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd4640cf8d0 con 0x7fd470102350 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd46ffff700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4600a2920 0x7fd4600a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd470076010 tx=0x7fd45c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd4753fd700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] conn(0x7fd4600ac100 0x7fd4600ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.217+0000 7fd4753fd700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] conn(0x7fd4600ac100 0x7fd4600ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:47.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.218+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== osd.10 v2:172.21.15.137:6824/293078987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd46401bec0 con 0x7fd4600ac100 2024-03-18T15:38:47.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.234+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 --> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd470062de0 con 0x7fd4600ac100 2024-03-18T15:38:47.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.234+0000 7fd46dffb700 1 -- 172.21.15.96:0/3344270370 <== osd.10 v2:172.21.15.137:6824/293078987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd470062de0 con 0x7fd4600ac100 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] conn(0x7fd4600ac100 msgr2=0x7fd4600ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] conn(0x7fd4600ac100 0x7fd4600ae520 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4600a2920 msgr2=0x7fd4600a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd4600a2920 0x7fd4600a4de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd470076010 tx=0x7fd45c005fb0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 msgr2=0x7fd4700745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 --2- 172.21.15.96:0/3344270370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd470102350 0x7fd4700745d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9a0 tx=0x7fd46400ecb0 comp rx=0 tx=0).stop 2024-03-18T15:38:47.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd4753fd700 1 -- 172.21.15.96:0/3344270370 reap_dead start 2024-03-18T15:38:47.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 shutdown_connections 2024-03-18T15:38:47.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 >> 172.21.15.96:0/3344270370 conn(0x7fd470079430 msgr2=0x7fd470100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:47.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 shutdown_connections 2024-03-18T15:38:47.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:47.235+0000 7fd476e60700 1 -- 172.21.15.96:0/3344270370 wait complete. 2024-03-18T15:38:48.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:48 smithi096 ceph-mon[31930]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:48.639 INFO:teuthology.orchestra.run.smithi096.stdout:292057776161 2024-03-18T15:38:48.639 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-18T15:38:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:48 smithi137 ceph-mon[35868]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:48.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:48 smithi193 ceph-mon[36702]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-18T15:38:49.401 INFO:teuthology.orchestra.run.smithi096.stdout:313532612634 2024-03-18T15:38:49.401 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-18T15:38:49.902 INFO:teuthology.orchestra.run.smithi096.stdout:77309411395 2024-03-18T15:38:49.903 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 -- 172.21.15.96:0/2382335163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9300107260 msgr2=0x7f9300107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 --2- 172.21.15.96:0/2382335163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9300107260 0x7f9300107640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009990 tx=0x7f92f0009ca0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 -- 172.21.15.96:0/2382335163 shutdown_connections 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 --2- 172.21.15.96:0/2382335163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9300101b70 0x7f930010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 --2- 172.21.15.96:0/2382335163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9300107260 0x7f9300107640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 --2- 172.21.15.96:0/2382335163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93001011b0 0x7f9300101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 -- 172.21.15.96:0/2382335163 >> 172.21.15.96:0/2382335163 conn(0x7f93000781e0 msgr2=0x7f93000785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:50.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 -- 172.21.15.96:0/2382335163 shutdown_connections 2024-03-18T15:38:50.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.390+0000 7f9305ebc700 1 -- 172.21.15.96:0/2382335163 wait complete. 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 Processor -- start 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 -- start start 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 0x7f9300102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9300101b70 0x7f93001033a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9300107260 0x7f93001038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93001114a0 con 0x7f93001011b0 2024-03-18T15:38:50.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9300111320 con 0x7f9300107260 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f9305ebc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9300111620 con 0x7f9300101b70 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f92ff7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 0x7f9300102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f92feffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9300101b70 0x7f93001033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f92ff7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 0x7f9300102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44066/0 (socket says 172.21.15.96:44066) 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f92ff7fe700 1 -- 172.21.15.96:0/270442821 learned_addr learned my addr 172.21.15.96:0/270442821 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.391+0000 7f92fffff700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9300107260 0x7f93001038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 -- 172.21.15.96:0/270442821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9300101b70 msgr2=0x7f93001033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9300101b70 0x7f93001033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 -- 172.21.15.96:0/270442821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9300107260 msgr2=0x7f93001038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9300107260 0x7f93001038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93001a6ed0 con 0x7f93001011b0 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92feffd700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9300101b70 0x7f93001033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92ff7fe700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 0x7f9300102e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0027070 con 0x7f93001011b0 2024-03-18T15:38:50.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f0003d70 con 0x7f93001011b0 2024-03-18T15:38:50.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93001a7160 con 0x7f93001011b0 2024-03-18T15:38:50.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f000f460 con 0x7f93001011b0 2024-03-18T15:38:50.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.392+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93001a7640 con 0x7f93001011b0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.393+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93001052b0 con 0x7f93001011b0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.394+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f92f002a070 con 0x7f93001011b0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fcff9700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f92e80a2920 0x7f92e80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92feffd700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f92e80a2920 0x7f92e80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f92f00cfe20 con 0x7f93001011b0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fcff9700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] conn(0x7f92e80ac100 0x7f92e80ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92e80aebd0 con 0x7f92e80ac100 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f92f00d0300 con 0x7f93001011b0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fffff700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] conn(0x7f92e80ac100 0x7f92e80ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92feffd700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f92e80a2920 0x7f92e80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9300078750 tx=0x7f92f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.395+0000 7f92fffff700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] conn(0x7f92e80ac100 0x7f92e80ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.396+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== osd.6 v2:172.21.15.96:6818/3660469146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92e80aebd0 con 0x7f92e80ac100 2024-03-18T15:38:50.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.412+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 --> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93001028a0 con 0x7f92e80ac100 2024-03-18T15:38:50.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f92fcff9700 1 -- 172.21.15.96:0/270442821 <== osd.6 v2:172.21.15.96:6818/3660469146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f93001028a0 con 0x7f92e80ac100 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] conn(0x7f92e80ac100 msgr2=0x7f92e80ae520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] conn(0x7f92e80ac100 0x7f92e80ae520 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f92e80a2920 msgr2=0x7f92e80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f92e80a2920 0x7f92e80a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9300078750 tx=0x7f92f4009260 comp rx=0 tx=0).stop 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 msgr2=0x7f9300102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 --2- 172.21.15.96:0/270442821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93001011b0 0x7f9300102e60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f000bab0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f92fffff700 1 -- 172.21.15.96:0/270442821 reap_dead start 2024-03-18T15:38:50.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 shutdown_connections 2024-03-18T15:38:50.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 >> 172.21.15.96:0/270442821 conn(0x7f93000781e0 msgr2=0x7f9300100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:50.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 shutdown_connections 2024-03-18T15:38:50.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.413+0000 7f9305ebc700 1 -- 172.21.15.96:0/270442821 wait complete. 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 -- 172.21.15.96:0/1111541717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c101b70 msgr2=0x7f275c10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 --2- 172.21.15.96:0/1111541717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c101b70 0x7f275c10e050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f27440098f0 tx=0x7f2744009c00 comp rx=0 tx=0).stop 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 -- 172.21.15.96:0/1111541717 shutdown_connections 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 --2- 172.21.15.96:0/1111541717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c101b70 0x7f275c10e050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 --2- 172.21.15.96:0/1111541717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f275c107210 0x7f275c1075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 --2- 172.21.15.96:0/1111541717 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c1011d0 0x7f275c101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 -- 172.21.15.96:0/1111541717 >> 172.21.15.96:0/1111541717 conn(0x7f275c078100 msgr2=0x7f275c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 -- 172.21.15.96:0/1111541717 shutdown_connections 2024-03-18T15:38:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.581+0000 7f2762d47700 1 -- 172.21.15.96:0/1111541717 wait complete. 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 Processor -- start 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 -- start start 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c1011d0 0x7f275c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 0x7f275c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f275c107210 0x7f275c1a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f275c1114e0 con 0x7f275c101b70 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f275c111360 con 0x7f275c1011d0 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2762d47700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f275c111660 con 0x7f275c107210 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 0x7f275c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 0x7f275c199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44092/0 (socket says 172.21.15.96:44092) 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 -- 172.21.15.96:0/4098462076 learned_addr learned my addr 172.21.15.96:0/4098462076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f2760ae3700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c1011d0 0x7f275c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f27612e4700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f275c107210 0x7f275c1a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 -- 172.21.15.96:0/4098462076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f275c107210 msgr2=0x7f275c1a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f275c107210 0x7f275c1a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 -- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c1011d0 msgr2=0x7f275c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f275c1011d0 0x7f275c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.582+0000 7f275bfff700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c1a4f60 con 0x7f275c101b70 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f275bfff700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 0x7f275c199ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9c0 tx=0x7f274c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c004a30 con 0x7f275c101b70 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c1a5190 con 0x7f275c101b70 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f274c00cca0 con 0x7f275c101b70 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c005610 con 0x7f275c101b70 2024-03-18T15:38:50.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.583+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a5720 con 0x7f275c101b70 2024-03-18T15:38:50.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.584+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f275c0629e0 con 0x7f275c101b70 2024-03-18T15:38:50.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.585+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f274c004cf0 con 0x7f275c101b70 2024-03-18T15:38:50.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.585+0000 7f2759ffb700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27480a29a0 0x7f27480a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2760ae3700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27480a29a0 0x7f27480a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f274c0ceef0 con 0x7f275c101b70 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2759ffb700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] conn(0x7f27480ac180 0x7f27480ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f274c00ce50 con 0x7f27480ac180 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f274c0cf3d0 con 0x7f275c101b70 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f2760ae3700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27480a29a0 0x7f27480a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f275400f540 tx=0x7f275400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f27612e4700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] conn(0x7f27480ac180 0x7f27480ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:50.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.586+0000 7f27612e4700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] conn(0x7f27480ac180 0x7f27480ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:50.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.587+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== osd.1 v2:172.21.15.137:6800/3508407501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f274c00ce50 con 0x7f27480ac180 2024-03-18T15:38:50.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.603+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 --> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f275c062de0 con 0x7f27480ac180 2024-03-18T15:38:50.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.603+0000 7f2759ffb700 1 -- 172.21.15.96:0/4098462076 <== osd.1 v2:172.21.15.137:6800/3508407501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f275c062de0 con 0x7f27480ac180 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] conn(0x7f27480ac180 msgr2=0x7f27480ae5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] conn(0x7f27480ac180 0x7f27480ae5a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27480a29a0 msgr2=0x7f27480a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27480a29a0 0x7f27480a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f275400f540 tx=0x7f275400b040 comp rx=0 tx=0).stop 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 msgr2=0x7f275c199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 --2- 172.21.15.96:0/4098462076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f275c101b70 0x7f275c199ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f274c00e9c0 tx=0x7f274c00ecd0 comp rx=0 tx=0).stop 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f27612e4700 1 -- 172.21.15.96:0/4098462076 reap_dead start 2024-03-18T15:38:50.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 shutdown_connections 2024-03-18T15:38:50.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 >> 172.21.15.96:0/4098462076 conn(0x7f275c078100 msgr2=0x7f275c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:50.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 shutdown_connections 2024-03-18T15:38:50.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:50.604+0000 7f2762d47700 1 -- 172.21.15.96:0/4098462076 wait complete. 2024-03-18T15:38:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:50 smithi137 ceph-mon[35868]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:50.886 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:50 smithi096 ceph-mon[31930]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:50.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:50 smithi193 ceph-mon[36702]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 170 B/s wr, 11 op/s 2024-03-18T15:38:51.223 INFO:teuthology.orchestra.run.smithi096.stdout:347892350998 2024-03-18T15:38:51.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-18T15:38:51.775 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:52.303 INFO:teuthology.orchestra.run.smithi096.stdout:107374182462 2024-03-18T15:38:52.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-18T15:38:52.644 INFO:teuthology.orchestra.run.smithi096.stdout:55834574922 2024-03-18T15:38:52.645 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-18T15:38:52.774 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:52 smithi096 ceph-mon[31930]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:38:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:52 smithi137 ceph-mon[35868]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:38:52.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:52 smithi193 ceph-mon[36702]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:38:53.039 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:53.046 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:53.051 INFO:teuthology.orchestra.run.smithi096.stdout:201863462961 2024-03-18T15:38:53.052 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-18T15:38:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:54 smithi096 ceph-mon[31930]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:38:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:54 smithi137 ceph-mon[35868]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:38:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:54 smithi193 ceph-mon[36702]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:38:55.169 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 -- 172.21.15.96:0/2903503550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c102d40 msgr2=0x7f528c10f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 --2- 172.21.15.96:0/2903503550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c102d40 0x7f528c10f130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f528400b0a0 tx=0x7f528400b3b0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 -- 172.21.15.96:0/2903503550 shutdown_connections 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 --2- 172.21.15.96:0/2903503550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c102d40 0x7f528c10f130 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 --2- 172.21.15.96:0/2903503550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c1083f0 0x7f528c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 --2- 172.21.15.96:0/2903503550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 0x7f528c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 -- 172.21.15.96:0/2903503550 >> 172.21.15.96:0/2903503550 conn(0x7f528c0793a0 msgr2=0x7f528c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 -- 172.21.15.96:0/2903503550 shutdown_connections 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.466+0000 7f5292d1b700 1 -- 172.21.15.96:0/2903503550 wait complete. 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 Processor -- start 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 -- start start 2024-03-18T15:38:56.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 0x7f528c0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 0x7f528c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f528c112610 con 0x7f528c1083f0 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f528c112490 con 0x7f528c102d40 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5292d1b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f528c112790 con 0x7f528c102380 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37586/0 (socket says 172.21.15.96:37586) 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 -- 172.21.15.96:0/1747403701 learned_addr learned my addr 172.21.15.96:0/1747403701 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f5290ab7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 0x7f528c0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f528bfff700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 0x7f528c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 -- 172.21.15.96:0/1747403701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 msgr2=0x7f528c0745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 0x7f528c0745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 -- 172.21.15.96:0/1747403701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 msgr2=0x7f528c074b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:56.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 0x7f528c074b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f52912b8700 1 -- 172.21.15.96:0/1747403701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1a6f70 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.467+0000 7f528bfff700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 0x7f528c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f52912b8700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f528400b780 tx=0x7f5284004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284021070 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c1a7200 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5284007b80 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52840052a0 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.468+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1a7700 con 0x7f528c1083f0 2024-03-18T15:38:56.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.469+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528c068f90 con 0x7f528c1083f0 2024-03-18T15:38:56.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.470+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f5284024070 con 0x7f528c1083f0 2024-03-18T15:38:56.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.471+0000 7f5289ffb700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 0x7f527c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:56.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.471+0000 7f5290ab7700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 0x7f527c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:56.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.471+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f528400f020 con 0x7f528c1083f0 2024-03-18T15:38:56.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.471+0000 7f5290ab7700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 0x7f527c0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5280009be0 tx=0x7f5280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:56.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.474+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f528402b020 con 0x7f528c1083f0 2024-03-18T15:38:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.506 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.508 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.508 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:56.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.706+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f528c02db70 con 0x7f528c1083f0 2024-03-18T15:38:56.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.706+0000 7f5289ffb700 1 -- 172.21.15.96:0/1747403701 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5284007d30 con 0x7f528c1083f0 2024-03-18T15:38:56.707 INFO:teuthology.orchestra.run.smithi096.stdout:141733920828 2024-03-18T15:38:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.708+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 msgr2=0x7f527c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.708+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 0x7f527c0a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5280009be0 tx=0x7f5280009260 comp rx=0 tx=0).stop 2024-03-18T15:38:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.708+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 msgr2=0x7f528c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:56.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.708+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f528400b780 tx=0x7f5284004d00 comp rx=0 tx=0).stop 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 shutdown_connections 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f527c0a2920 0x7f527c0a4de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f528c1083f0 0x7f528c075050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f528c102380 0x7f528c0745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 --2- 172.21.15.96:0/1747403701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f528c102d40 0x7f528c074b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 >> 172.21.15.96:0/1747403701 conn(0x7f528c0793a0 msgr2=0x7f528c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 shutdown_connections 2024-03-18T15:38:56.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:56.709+0000 7f5292d1b700 1 -- 172.21.15.96:0/1747403701 wait complete. 2024-03-18T15:38:56.722 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:56 smithi096 ceph-mon[31930]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:38:56.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:56 smithi137 ceph-mon[35868]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:38:56.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:56 smithi193 ceph-mon[36702]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:38:57.659 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:57 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1747403701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 -- 172.21.15.96:0/565872706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c107210 msgr2=0x7fd78c1075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 --2- 172.21.15.96:0/565872706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c107210 0x7fd78c1075f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b0a0 tx=0x7fd77c00b3b0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 -- 172.21.15.96:0/565872706 shutdown_connections 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 --2- 172.21.15.96:0/565872706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c10e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 --2- 172.21.15.96:0/565872706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c107210 0x7fd78c1075f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 --2- 172.21.15.96:0/565872706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c1011d0 0x7fd78c101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 -- 172.21.15.96:0/565872706 >> 172.21.15.96:0/565872706 conn(0x7fd78c078100 msgr2=0x7fd78c078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 -- 172.21.15.96:0/565872706 shutdown_connections 2024-03-18T15:38:57.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 -- 172.21.15.96:0/565872706 wait complete. 2024-03-18T15:38:57.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.760+0000 7fd790989700 1 Processor -- start 2024-03-18T15:38:57.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 -- start start 2024-03-18T15:38:57.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c1011d0 0x7fd78c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:57.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c107210 0x7fd78c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c111470 con 0x7fd78c1011d0 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1112f0 con 0x7fd78c107210 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd790989700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1115f0 con 0x7fd78c101b70 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c1011d0 0x7fd78c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:34204/0 (socket says 172.21.15.96:34204) 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 -- 172.21.15.96:0/3875936082 learned_addr learned my addr 172.21.15.96:0/3875936082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd78ad9d700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c107210 0x7fd78c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 -- 172.21.15.96:0/3875936082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c107210 msgr2=0x7fd78c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c107210 0x7fd78c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 -- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c1011d0 msgr2=0x7fd78c1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c1011d0 0x7fd78c1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.761+0000 7fd781d9b700 1 -- 172.21.15.96:0/3875936082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c19b230 con 0x7fd78c101b70 2024-03-18T15:38:57.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd781d9b700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9a0 tx=0x7fd77400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400ca10 con 0x7fd78c101b70 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77400cbb0 con 0x7fd78c101b70 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c19b520 con 0x7fd78c101b70 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774018680 con 0x7fd78c101b70 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.762+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a77e0 con 0x7fd78c101b70 2024-03-18T15:38:57.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.763+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78c068f90 con 0x7fd78c101b70 2024-03-18T15:38:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.764+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fd774010430 con 0x7fd78c101b70 2024-03-18T15:38:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.764+0000 7fd783fff700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 0x7fd7780a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:38:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.765+0000 7fd78a59c700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 0x7fd7780a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:38:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.765+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd7740cf210 con 0x7fd78c101b70 2024-03-18T15:38:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.765+0000 7fd78a59c700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 0x7fd7780a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009e60 tx=0x7fd77c00b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:38:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.768+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd77409c200 con 0x7fd78c101b70 2024-03-18T15:38:57.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:57 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1747403701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T15:38:57.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:57 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1747403701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-18T15:38:57.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.976+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd78c1a7a10 con 0x7fd78c101b70 2024-03-18T15:38:57.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.976+0000 7fd783fff700 1 -- 172.21.15.96:0/3875936082 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd77409bb00 con 0x7fd78c101b70 2024-03-18T15:38:57.977 INFO:teuthology.orchestra.run.smithi096.stdout:257698037800 2024-03-18T15:38:57.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.978+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 msgr2=0x7fd7780a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:57.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.978+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 0x7fd7780a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009e60 tx=0x7fd77c00b700 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 msgr2=0x7fd78c1a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd77400e9a0 tx=0x7fd77400ecb0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 shutdown_connections 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7780a2920 0x7fd7780a4de0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd78c107210 0x7fd78c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd78c1011d0 0x7fd78c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 --2- 172.21.15.96:0/3875936082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd78c101b70 0x7fd78c1a0e50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 >> 172.21.15.96:0/3875936082 conn(0x7fd78c078100 msgr2=0x7fd78c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 shutdown_connections 2024-03-18T15:38:57.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:38:57.979+0000 7fd790989700 1 -- 172.21.15.96:0/3875936082 wait complete. 2024-03-18T15:38:58.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920828 for osd.4 2024-03-18T15:38:58.224 DEBUG:teuthology.parallel:result is None 2024-03-18T15:38:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:58 smithi096 ceph-mon[31930]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:38:58.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:38:58 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3875936082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T15:38:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:58 smithi137 ceph-mon[35868]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:38:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:38:58 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3875936082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T15:38:58.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037796 got 257698037800 for osd.8 2024-03-18T15:38:58.889 DEBUG:teuthology.parallel:result is None 2024-03-18T15:38:58.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:58 smithi193 ceph-mon[36702]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:38:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:38:58 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3875936082' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-18T15:38:59.427 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:59.428 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:38:59.774 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 -- 172.21.15.96:0/3572463120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102510 msgr2=0x7f8b2c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 --2- 172.21.15.96:0/3572463120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102510 0x7f8b2c1028f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b280099c0 tx=0x7f8b28009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 -- 172.21.15.96:0/3572463120 shutdown_connections 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 --2- 172.21.15.96:0/3572463120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c1075f0 0x7f8b2c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 --2- 172.21.15.96:0/3572463120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102510 0x7f8b2c1028f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 --2- 172.21.15.96:0/3572463120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 -- 172.21.15.96:0/3572463120 >> 172.21.15.96:0/3572463120 conn(0x7f8b2c0fdd80 msgr2=0x7f8b2c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 -- 172.21.15.96:0/3572463120 shutdown_connections 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.088+0000 7f8b33d17700 1 -- 172.21.15.96:0/3572463120 wait complete. 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 Processor -- start 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 -- start start 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:00.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c1075f0 0x7f8b2c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c1125f0 con 0x7f8b2c102ec0 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112470 con 0x7f8b2c102510 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b33d17700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c112770 con 0x7f8b2c1075f0 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b312b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b31ab3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b312b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37612/0 (socket says 172.21.15.96:37612) 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b31ab3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:50054/0 (socket says 172.21.15.96:50054) 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b312b2700 1 -- 172.21.15.96:0/1134379036 learned_addr learned my addr 172.21.15.96:0/1134379036 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.089+0000 7f8b322b4700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c1075f0 0x7f8b2c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 -- 172.21.15.96:0/1134379036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c1075f0 msgr2=0x7f8b2c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c1075f0 0x7f8b2c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 -- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 msgr2=0x7f8b2c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 -- 172.21.15.96:0/1134379036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c198ff0 con 0x7f8b2c102510 2024-03-18T15:39:00.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b312b2700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b31ab3700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b28005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28027070 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28003680 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c199280 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28022e10 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.090+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c1a55a0 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.091+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c068f90 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.092+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8b2802a070 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.093+0000 7f8b22ffd700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 0x7f8b180a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.093+0000 7f8b312b2700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 0x7f8b180a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.093+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8b280cf720 con 0x7f8b2c102510 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.093+0000 7f8b312b2700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 0x7f8b180a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ff1a0 tx=0x7f8b1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:00.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.097+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8b28013070 con 0x7f8b2c102510 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stdout:223338299438 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.305+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8b2c02db70 con 0x7f8b2c102510 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.306+0000 7f8b22ffd700 1 -- 172.21.15.96:0/1134379036 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8b2809c8c0 con 0x7f8b2c102510 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 msgr2=0x7f8b180a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 0x7f8b180a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0ff1a0 tx=0x7f8b1c009260 comp rx=0 tx=0).stop 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 msgr2=0x7f8b2c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b280098e0 tx=0x7f8b28005f10 comp rx=0 tx=0).stop 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 shutdown_connections 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8b180a2920 0x7f8b180a4de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b2c1075f0 0x7f8b2c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b2c102510 0x7f8b2c19e6d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 --2- 172.21.15.96:0/1134379036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b2c102ec0 0x7f8b2c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:00.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 >> 172.21.15.96:0/1134379036 conn(0x7f8b2c0fdd80 msgr2=0x7f8b2c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:00.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 shutdown_connections 2024-03-18T15:39:00.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:00.308+0000 7f8b33d17700 1 -- 172.21.15.96:0/1134379036 wait complete. 2024-03-18T15:39:00.837 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:00 smithi096 ceph-mon[31930]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-03-18T15:39:00.837 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:00 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1134379036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T15:39:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:00 smithi137 ceph-mon[35868]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-03-18T15:39:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:00 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1134379036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T15:39:00.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:00 smithi193 ceph-mon[36702]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 9 op/s 2024-03-18T15:39:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:00 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1134379036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.847 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.848 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T15:39:01.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]: dispatch 2024-03-18T15:39:01.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-18T15:39:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T15:39:02.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:02 smithi137 ceph-mon[35868]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-18T15:39:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-18T15:39:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-18T15:39:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-18T15:39:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T15:39:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:02 smithi193 ceph-mon[36702]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [5, 7]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.22", "id": [3, 9]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-18T15:39:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:02 smithi096 ceph-mon[31930]: osdmap e87: 12 total, 12 up, 12 in 2024-03-18T15:39:03.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:03 smithi096 ceph-mon[31930]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T15:39:03.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:03 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:39:03.823 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:03 smithi193 ceph-mon[36702]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T15:39:03.823 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:03 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:39:03.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:03 smithi137 ceph-mon[35868]: osdmap e88: 12 total, 12 up, 12 in 2024-03-18T15:39:03.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:03 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:39:04.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:04 smithi137 ceph-mon[35868]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-18T15:39:04.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:04 smithi193 ceph-mon[36702]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-18T15:39:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:04 smithi096 ceph-mon[31930]: pgmap v319: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-03-18T15:39:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:05 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:05 smithi193 ceph-mon[36702]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-18T15:39:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:05 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:05 smithi096 ceph-mon[31930]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-18T15:39:06.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:06.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:06.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:06.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:05 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:06.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:05 smithi137 ceph-mon[35868]: pgmap v320: 97 pgs: 5 peering, 92 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 0 B/s, 0 objects/s recovering 2024-03-18T15:39:06.122 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299438 for osd.7 2024-03-18T15:39:06.123 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:08.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 -- 172.21.15.96:0/1052625311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 msgr2=0x7f90e4105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 --2- 172.21.15.96:0/1052625311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e4105dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f90d80098f0 tx=0x7f90d8009c00 comp rx=0 tx=0).stop 2024-03-18T15:39:08.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 -- 172.21.15.96:0/1052625311 shutdown_connections 2024-03-18T15:39:08.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 --2- 172.21.15.96:0/1052625311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e41064b0 0x7f90e410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 --2- 172.21.15.96:0/1052625311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e4101460 0x7f90e4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 --2- 172.21.15.96:0/1052625311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e4105dd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 -- 172.21.15.96:0/1052625311 >> 172.21.15.96:0/1052625311 conn(0x7f90e4077220 msgr2=0x7f90e4077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 -- 172.21.15.96:0/1052625311 shutdown_connections 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.172+0000 7f90e8855700 1 -- 172.21.15.96:0/1052625311 wait complete. 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 Processor -- start 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 -- start start 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e41064b0 0x7f90e4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4111490 con 0x7f90e41064b0 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4111310 con 0x7f90e4101e10 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e8855700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4111610 con 0x7f90e4101460 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e2d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e41064b0 0x7f90e4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e259c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:41674/0 (socket says 172.21.15.96:41674) 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e259c700 1 -- 172.21.15.96:0/1102081602 learned_addr learned my addr 172.21.15.96:0/1102081602 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.173+0000 7f90e1d9b700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 -- 172.21.15.96:0/1102081602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 msgr2=0x7f90e419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 -- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e41064b0 msgr2=0x7f90e4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e41064b0 0x7f90e4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 -- 172.21.15.96:0/1102081602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e4198030 con 0x7f90e4101e10 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e259c700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e1d9b700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e419dc10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009230 tx=0x7f90d800ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d800f8a0 con 0x7f90e4101e10 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d800fa40 con 0x7f90e4101e10 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d8003680 con 0x7f90e4101e10 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e41982c0 con 0x7f90e4101e10 2024-03-18T15:39:08.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.174+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e406a810 con 0x7f90e4101e10 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.175+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e41028b0 con 0x7f90e4101e10 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.176+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f90d8005040 con 0x7f90e4101e10 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.177+0000 7f90d37fe700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 0x7f90cc0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.177+0000 7f90e259c700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 0x7f90cc0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.177+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f90d80d3bd0 con 0x7f90e4101e10 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.177+0000 7f90e259c700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 0x7f90cc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009750 tx=0x7f90d4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:08.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.180+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f90d8013030 con 0x7f90e4101e10 2024-03-18T15:39:08.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.394+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f90e4198d80 con 0x7f90e4101e10 2024-03-18T15:39:08.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.395+0000 7f90d37fe700 1 -- 172.21.15.96:0/1102081602 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90d80a0b30 con 0x7f90e4101e10 2024-03-18T15:39:08.396 INFO:teuthology.orchestra.run.smithi096.stdout:77309411400 2024-03-18T15:39:08.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 msgr2=0x7f90cc0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 0x7f90cc0a4e60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009750 tx=0x7f90d4010040 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 msgr2=0x7f90e419dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e419dc10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009230 tx=0x7f90d800ff20 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 shutdown_connections 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90cc0a29a0 0x7f90cc0a4e60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90e41064b0 0x7f90e4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e4101460 0x7f90e419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 --2- 172.21.15.96:0/1102081602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90e4101e10 0x7f90e419dc10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 >> 172.21.15.96:0/1102081602 conn(0x7f90e4077220 msgr2=0x7f90e40ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 shutdown_connections 2024-03-18T15:39:08.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.397+0000 7f90e8855700 1 -- 172.21.15.96:0/1102081602 wait complete. 2024-03-18T15:39:08.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:08 smithi096 ceph-mon[31930]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:08.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:08 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3370178724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T15:39:08.667 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:08 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1102081602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T15:39:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:08 smithi137 ceph-mon[35868]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:08 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3370178724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T15:39:08.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:08 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1102081602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T15:39:08.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 -- 172.21.15.96:0/265847716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3201075e0 msgr2=0x7fb32010f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 --2- 172.21.15.96:0/265847716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3201075e0 0x7fb32010f190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb31800b0a0 tx=0x7fb31800b3b0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 -- 172.21.15.96:0/265847716 shutdown_connections 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 --2- 172.21.15.96:0/265847716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3201075e0 0x7fb32010f190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 --2- 172.21.15.96:0/265847716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006aa70 0x7fb32006ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 --2- 172.21.15.96:0/265847716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb32006b420 0x7fb320106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 -- 172.21.15.96:0/265847716 >> 172.21.15.96:0/265847716 conn(0x7fb320077220 msgr2=0x7fb320077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 -- 172.21.15.96:0/265847716 shutdown_connections 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.079+0000 7fb325fe2700 1 -- 172.21.15.96:0/265847716 wait complete. 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 Processor -- start 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 -- start start 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006b420 0x7fb32019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3201075e0 0x7fb320198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3201125a0 con 0x7fb32006aa70 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb320112420 con 0x7fb3201075e0 2024-03-18T15:39:08.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.080+0000 7fb325fe2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb320112720 con 0x7fb32006b420 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50258/0 (socket says 172.21.15.96:50258) 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 -- 172.21.15.96:0/3370178724 learned_addr learned my addr 172.21.15.96:0/3370178724 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stdout:107374182467 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31effd700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006b420 0x7fb32019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31ffff700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3201075e0 0x7fb320198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 -- 172.21.15.96:0/3370178724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006b420 msgr2=0x7fb32019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006b420 0x7fb32019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 -- 172.21.15.96:0/3370178724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3201075e0 msgr2=0x7fb320198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3201075e0 0x7fb320198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 -- 172.21.15.96:0/3370178724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb320199020 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31f7fe700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e9a0 tx=0x7fb31000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31000ca10 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb320199310 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb31000cbb0 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310018680 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.081+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3201a55f0 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.082+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3201039e0 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.083+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb310010430 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.084+0000 7fb31cff9700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 0x7fb3080a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.084+0000 7fb31effd700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 0x7fb3080a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.084+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb3100cf2d0 con 0x7fb32006aa70 2024-03-18T15:39:08.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.084+0000 7fb31effd700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 0x7fb3080a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb320199f00 tx=0x7fb314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.088+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb31009c200 con 0x7fb32006aa70 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.299+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb3201a57c0 con 0x7fb32006aa70 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.299+0000 7fb31cff9700 1 -- 172.21.15.96:0/3370178724 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb31009bb00 con 0x7fb32006aa70 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 msgr2=0x7fb3080a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 0x7fb3080a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb320199f00 tx=0x7fb314009260 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 msgr2=0x7fb32019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb31000e9a0 tx=0x7fb31000ecb0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 shutdown_connections 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb3080a2920 0x7fb3080a4de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3201075e0 0x7fb320198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb32006aa70 0x7fb32019e670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 --2- 172.21.15.96:0/3370178724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb32006b420 0x7fb32019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 >> 172.21.15.96:0/3370178724 conn(0x7fb320077220 msgr2=0x7fb3200ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 shutdown_connections 2024-03-18T15:39:08.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:08.301+0000 7fb325fe2700 1 -- 172.21.15.96:0/3370178724 wait complete. 2024-03-18T15:39:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:08 smithi193 ceph-mon[36702]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:08 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3370178724' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-18T15:39:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:08 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1102081602' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-18T15:39:09.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182467 for osd.3 2024-03-18T15:39:09.543 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 -- 172.21.15.96:0/3019666776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 msgr2=0x7fce08105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/3019666776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce08105dd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0099c0 tx=0x7fcdfc009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 -- 172.21.15.96:0/3019666776 shutdown_connections 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/3019666776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 0x7fce0810e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/3019666776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce08101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/3019666776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce08105dd0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.566+0000 7fce0d3f3700 1 -- 172.21.15.96:0/3019666776 >> 172.21.15.96:0/3019666776 conn(0x7fce08077220 msgr2=0x7fce08077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 -- 172.21.15.96:0/3019666776 shutdown_connections 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 -- 172.21.15.96:0/3019666776 wait complete. 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 Processor -- start 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 -- start start 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce0819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.567+0000 7fce0d3f3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 0x7fce08198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce0d3f3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce08111470 con 0x7fce08101e10 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce0d3f3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce081112f0 con 0x7fce08101460 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce0d3f3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce081115f0 con 0x7fce081064b0 2024-03-18T15:39:10.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:56618/0 (socket says 172.21.15.96:56618) 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 -- 172.21.15.96:0/1861948631 learned_addr learned my addr 172.21.15.96:0/1861948631 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce077fe700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 0x7fce08198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce067fc700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce0819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 -- 172.21.15.96:0/1861948631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 msgr2=0x7fce08198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 0x7fce08198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 -- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 msgr2=0x7fce0819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce0819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 -- 172.21.15.96:0/1861948631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce08199060 con 0x7fce08101460 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce067fc700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce0819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce06ffd700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e9a0 tx=0x7fcdf800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:10.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800cbd0 con 0x7fce08101460 2024-03-18T15:39:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdf800cd70 con 0x7fce08101460 2024-03-18T15:39:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce08199350 con 0x7fce08101460 2024-03-18T15:39:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8018750 con 0x7fce08101460 2024-03-18T15:39:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.568+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081a5610 con 0x7fce08101460 2024-03-18T15:39:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.569+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08068f90 con 0x7fce08101460 2024-03-18T15:39:10.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.570+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fcdf8010430 con 0x7fce08101460 2024-03-18T15:39:10.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.571+0000 7fcdeffff700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 0x7fcdf00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:10.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.571+0000 7fce067fc700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 0x7fcdf00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:10.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.571+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fcdf80ceec0 con 0x7fce08101460 2024-03-18T15:39:10.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.571+0000 7fce067fc700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 0x7fcdf00a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:10.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.576+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fcdf809bdf0 con 0x7fce08101460 2024-03-18T15:39:10.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.783+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fce081a57e0 con 0x7fce08101460 2024-03-18T15:39:10.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.784+0000 7fcdeffff700 1 -- 172.21.15.96:0/1861948631 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcdf809b6f0 con 0x7fce08101460 2024-03-18T15:39:10.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:10 smithi096 ceph-mon[31930]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:10.785 INFO:teuthology.orchestra.run.smithi096.stdout:201863462965 2024-03-18T15:39:10.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 msgr2=0x7fcdf00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:10.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 0x7fcdf00a4de0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc000c00 tx=0x7fcdfc005ab0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 msgr2=0x7fce0819e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e9a0 tx=0x7fcdf800ecb0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 shutdown_connections 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcdf00a2920 0x7fcdf00a4de0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce081064b0 0x7fce08198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce08101460 0x7fce0819e740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 --2- 172.21.15.96:0/1861948631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce08101e10 0x7fce0819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.786+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 >> 172.21.15.96:0/1861948631 conn(0x7fce08077220 msgr2=0x7fce08105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.787+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 shutdown_connections 2024-03-18T15:39:10.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:10.787+0000 7fce0d3f3700 1 -- 172.21.15.96:0/1861948631 wait complete. 2024-03-18T15:39:10.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:10 smithi137 ceph-mon[35868]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:10 smithi193 ceph-mon[36702]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:11.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.214+0000 7f7124b24700 1 -- 172.21.15.96:0/1155578382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 msgr2=0x7f712010c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.214+0000 7f7124b24700 1 --2- 172.21.15.96:0/1155578382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712010c3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7110009990 tx=0x7f7110009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.214+0000 7f7124b24700 1 -- 172.21.15.96:0/1155578382 shutdown_connections 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.214+0000 7f7124b24700 1 --2- 172.21.15.96:0/1155578382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010c9c0 0x7f712010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 --2- 172.21.15.96:0/1155578382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71201035e0 0x7f71201039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 --2- 172.21.15.96:0/1155578382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712010c3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 -- 172.21.15.96:0/1155578382 >> 172.21.15.96:0/1155578382 conn(0x7f7120078300 msgr2=0x7f7120078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 -- 172.21.15.96:0/1155578382 shutdown_connections 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 -- 172.21.15.96:0/1155578382 wait complete. 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 Processor -- start 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 -- start start 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71201035e0 0x7f712019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.215+0000 7f7124b24700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:11.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712010c9c0 0x7f71201989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71201126a0 con 0x7f7120103f00 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112520 con 0x7f712010c9c0 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7120112820 con 0x7f71201035e0 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50290/0 (socket says 172.21.15.96:50290) 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 -- 172.21.15.96:0/159452111 learned_addr learned my addr 172.21.15.96:0/159452111 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711e59c700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71201035e0 0x7f712019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711ed9d700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712010c9c0 0x7f71201989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 -- 172.21.15.96:0/159452111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71201035e0 msgr2=0x7f712019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71201035e0 0x7f712019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 -- 172.21.15.96:0/159452111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712010c9c0 msgr2=0x7f71201989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712010c9c0 0x7f71201989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 -- 172.21.15.96:0/159452111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7120199100 con 0x7f7120103f00 2024-03-18T15:39:11.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f711dd9b700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7110000c00 tx=0x7f711000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110027070 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7120199390 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110003d70 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a5660 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.216+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000f460 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.217+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120068f90 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.219+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f711002a070 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.219+0000 7f71177fe700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 0x7f710c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.220+0000 7f711e59c700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 0x7f710c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.220+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f71100cfd70 con 0x7f7120103f00 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.220+0000 7f711e59c700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 0x7f710c0a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7120078a30 tx=0x7f7108009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:11.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.223+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7110013070 con 0x7f7120103f00 2024-03-18T15:39:11.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.421+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f71201a5890 con 0x7f7120103f00 2024-03-18T15:39:11.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.422+0000 7f71177fe700 1 -- 172.21.15.96:0/159452111 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f711009cd30 con 0x7f7120103f00 2024-03-18T15:39:11.423 INFO:teuthology.orchestra.run.smithi096.stdout:34359738450 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 msgr2=0x7f710c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 0x7f710c0a4de0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7120078a30 tx=0x7f7108009260 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 msgr2=0x7f712019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7110000c00 tx=0x7f711000bab0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 shutdown_connections 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f710c0a2920 0x7f710c0a4de0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712010c9c0 0x7f71201989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71201035e0 0x7f712019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 --2- 172.21.15.96:0/159452111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7120103f00 0x7f712019ecd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 >> 172.21.15.96:0/159452111 conn(0x7f7120078300 msgr2=0x7f71200fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 shutdown_connections 2024-03-18T15:39:11.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:11.424+0000 7f7124b24700 1 -- 172.21.15.96:0/159452111 wait complete. 2024-03-18T15:39:11.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:11 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1861948631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T15:39:11.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:11 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/159452111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T15:39:11.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:11 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1861948631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T15:39:11.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:11 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/159452111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T15:39:11.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:11 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1861948631' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-18T15:39:11.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:11 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/159452111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-18T15:39:12.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:12 smithi096 ceph-mon[31930]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:12 smithi137 ceph-mon[35868]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:12 smithi193 ceph-mon[36702]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:13.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 -- 172.21.15.96:0/864807302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 msgr2=0x7f49d8074ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:13.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 --2- 172.21.15.96:0/864807302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d8074ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009990 tx=0x7f49cc009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 -- 172.21.15.96:0/864807302 shutdown_connections 2024-03-18T15:39:13.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 --2- 172.21.15.96:0/864807302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 0x7f49d8115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 --2- 172.21.15.96:0/864807302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 0x7f49d8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 --2- 172.21.15.96:0/864807302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d8074ec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.255+0000 7f49de3d4700 1 -- 172.21.15.96:0/864807302 >> 172.21.15.96:0/864807302 conn(0x7f49d8079400 msgr2=0x7f49d8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- 172.21.15.96:0/864807302 shutdown_connections 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- 172.21.15.96:0/864807302 wait complete. 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 Processor -- start 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- start start 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 0x7f49d819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 0x7f49d81a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8118bb0 con 0x7f49d8074a40 2024-03-18T15:39:13.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8118a30 con 0x7f49d8112f60 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.256+0000 7f49de3d4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49d8118d30 con 0x7f49d8076300 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50300/0 (socket says 172.21.15.96:50300) 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 -- 172.21.15.96:0/566676925 learned_addr learned my addr 172.21.15.96:0/566676925 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d77fe700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 0x7f49d819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49dc971700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 0x7f49d81a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 -- 172.21.15.96:0/566676925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 msgr2=0x7f49d819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 0x7f49d819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 -- 172.21.15.96:0/566676925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 msgr2=0x7f49d81a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 0x7f49d81a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 -- 172.21.15.96:0/566676925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d81a4110 con 0x7f49d8074a40 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d7fff700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e9c0 tx=0x7f49c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:13.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c800cbd0 con 0x7f49d8074a40 2024-03-18T15:39:13.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d81a4400 con 0x7f49d8074a40 2024-03-18T15:39:13.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c800ee60 con 0x7f49d8074a40 2024-03-18T15:39:13.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c8005630 con 0x7f49d8074a40 2024-03-18T15:39:13.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.257+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d81abc80 con 0x7f49d8074a40 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.258+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d8068f90 con 0x7f49d8074a40 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.260+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f49c8004750 con 0x7f49d8074a40 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.260+0000 7f49d57fa700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 0x7f49c00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.260+0000 7f49d77fe700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 0x7f49c00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.260+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f49c80cf0b0 con 0x7f49d8074a40 2024-03-18T15:39:13.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.261+0000 7f49d77fe700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 0x7f49c00a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00b2e0 tx=0x7f49cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:13.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.264+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f49c8053740 con 0x7f49d8074a40 2024-03-18T15:39:13.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.471+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f49d81a0460 con 0x7f49d8074a40 2024-03-18T15:39:13.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.472+0000 7f49d57fa700 1 -- 172.21.15.96:0/566676925 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f49c809bfe0 con 0x7f49d8074a40 2024-03-18T15:39:13.473 INFO:teuthology.orchestra.run.smithi096.stdout:313532612639 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 msgr2=0x7f49c00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 0x7f49c00a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f49cc00b2e0 tx=0x7f49cc005960 comp rx=0 tx=0).stop 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 msgr2=0x7f49d819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e9c0 tx=0x7f49c800ecd0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 shutdown_connections 2024-03-18T15:39:13.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49c00a2920 0x7f49c00a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49d8112f60 0x7f49d81a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49d8074a40 0x7f49d819f100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 --2- 172.21.15.96:0/566676925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49d8076300 0x7f49d819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 >> 172.21.15.96:0/566676925 conn(0x7f49d8079400 msgr2=0x7f49d8111d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 shutdown_connections 2024-03-18T15:39:13.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:13.474+0000 7f49de3d4700 1 -- 172.21.15.96:0/566676925 wait complete. 2024-03-18T15:39:13.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:13 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/566676925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T15:39:13.903 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:13 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/566676925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T15:39:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:13 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/566676925' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-18T15:39:14.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 -- 172.21.15.96:0/740575130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d0069f80 msgr2=0x7fb2d006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 --2- 172.21.15.96:0/740575130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d0069f80 0x7fb2d006a3e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00099c0 tx=0x7fb2c0009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 -- 172.21.15.96:0/740575130 shutdown_connections 2024-03-18T15:39:14.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 --2- 172.21.15.96:0/740575130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d006a920 0x7fb2d010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 --2- 172.21.15.96:0/740575130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0108320 0x7fb2d0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 --2- 172.21.15.96:0/740575130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d0069f80 0x7fb2d006a3e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 -- 172.21.15.96:0/740575130 >> 172.21.15.96:0/740575130 conn(0x7fb2d0078100 msgr2=0x7fb2d0078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 -- 172.21.15.96:0/740575130 shutdown_connections 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.004+0000 7fb2d867a700 1 -- 172.21.15.96:0/740575130 wait complete. 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 Processor -- start 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 -- start start 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0069f80 0x7fb2d019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d0108320 0x7fb2d0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0112580 con 0x7fb2d006a920 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0112400 con 0x7fb2d0108320 2024-03-18T15:39:14.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d867a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0112700 con 0x7fb2d0069f80 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56496/0 (socket says 172.21.15.96:56496) 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 -- 172.21.15.96:0/2436973276 learned_addr learned my addr 172.21.15.96:0/2436973276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d6416700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0069f80 0x7fb2d019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d6c17700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d0108320 0x7fb2d0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 -- 172.21.15.96:0/2436973276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0069f80 msgr2=0x7fb2d019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0069f80 0x7fb2d019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 -- 172.21.15.96:0/2436973276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d0108320 msgr2=0x7fb2d0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d0108320 0x7fb2d0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 -- 172.21.15.96:0/2436973276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d01990c0 con 0x7fb2d006a920 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.005+0000 7fb2d5c15700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00059d0 tx=0x7fb2c000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.006+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0027070 con 0x7fb2d006a920 2024-03-18T15:39:14.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.006+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c0003680 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.006+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d0199350 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.006+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00173f0 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.006+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01a5620 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.007+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d01039d0 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.008+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fb2c002a070 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.008+0000 7fb2c77fe700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 0x7fb2bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.008+0000 7fb2d6416700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 0x7fb2bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.009+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb2c00cf710 con 0x7fb2d006a920 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.009+0000 7fb2d6416700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 0x7fb2bc0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0107e80 tx=0x7fb2cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.012+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2c0013070 con 0x7fb2d006a920 2024-03-18T15:39:14.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.215+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb2d002db70 con 0x7fb2d006a920 2024-03-18T15:39:14.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.215+0000 7fb2c77fe700 1 -- 172.21.15.96:0/2436973276 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2c009c6d0 con 0x7fb2d006a920 2024-03-18T15:39:14.216 INFO:teuthology.orchestra.run.smithi096.stdout:55834574926 2024-03-18T15:39:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.217+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 msgr2=0x7fb2bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.217+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 0x7fb2bc0a4de0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0107e80 tx=0x7fb2cc009260 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.217+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 msgr2=0x7fb2d019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.217+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00059d0 tx=0x7fb2c000fa00 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 shutdown_connections 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2bc0a2920 0x7fb2bc0a4de0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d0108320 0x7fb2d0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0069f80 0x7fb2d019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 --2- 172.21.15.96:0/2436973276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d006a920 0x7fb2d019ec90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 >> 172.21.15.96:0/2436973276 conn(0x7fb2d0078100 msgr2=0x7fb2d00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 shutdown_connections 2024-03-18T15:39:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.218+0000 7fb2d867a700 1 -- 172.21.15.96:0/2436973276 wait complete. 2024-03-18T15:39:14.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- 172.21.15.96:0/3276163115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc108300 msgr2=0x7fe2cc1086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3276163115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc108300 0x7fe2cc1086e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80099c0 tx=0x7fe2c8009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- 172.21.15.96:0/3276163115 shutdown_connections 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3276163115 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc10f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3276163115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc108300 0x7fe2cc1086e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3276163115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc102320 0x7fe2cc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- 172.21.15.96:0/3276163115 >> 172.21.15.96:0/3276163115 conn(0x7fe2cc079400 msgr2=0x7fe2cc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- 172.21.15.96:0/3276163115 shutdown_connections 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- 172.21.15.96:0/3276163115 wait complete. 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 Processor -- start 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.664+0000 7fe2d424f700 1 -- start start 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102320 0x7fe2cc0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc108300 0x7fe2cc075050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc1125b0 con 0x7fe2cc102ce0 2024-03-18T15:39:14.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc112430 con 0x7fe2cc108300 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc112730 con 0x7fe2cc102320 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56500/0 (socket says 172.21.15.96:56500) 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 -- 172.21.15.96:0/3257744449 learned_addr learned my addr 172.21.15.96:0/3257744449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d1feb700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102320 0x7fe2cc0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d27ec700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc108300 0x7fe2cc075050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 -- 172.21.15.96:0/3257744449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102320 msgr2=0x7fe2cc0745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102320 0x7fe2cc0745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 -- 172.21.15.96:0/3257744449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc108300 msgr2=0x7fe2cc075050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc108300 0x7fe2cc075050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 -- 172.21.15.96:0/3257744449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc1a6ed0 con 0x7fe2cc102ce0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d17ea700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00e9a0 tx=0x7fe2bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00ca10 con 0x7fe2cc102ce0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2bc00cbb0 con 0x7fe2cc102ce0 2024-03-18T15:39:14.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc1a71c0 con 0x7fe2cc102ce0 2024-03-18T15:39:14.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc018680 con 0x7fe2cc102ce0 2024-03-18T15:39:14.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.665+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc1a7700 con 0x7fe2cc102ce0 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.666+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2cc068f90 con 0x7fe2cc102ce0 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.668+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fe2bc010430 con 0x7fe2cc102ce0 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.669+0000 7fe2c2ffd700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 0x7fe2b80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.669+0000 7fe2d1feb700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 0x7fe2b80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.669+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe2bc0cf2d0 con 0x7fe2cc102ce0 2024-03-18T15:39:14.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.669+0000 7fe2d1feb700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 0x7fe2b80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000c00 tx=0x7fe2c8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.673+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe2bc09c200 con 0x7fe2cc102ce0 2024-03-18T15:39:14.789 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:14 smithi096 ceph-mon[31930]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:14.789 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:14 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2436973276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T15:39:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:14 smithi137 ceph-mon[35868]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:14 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2436973276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T15:39:14.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.873+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe2cc02db70 con 0x7fe2cc102ce0 2024-03-18T15:39:14.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.874+0000 7fe2c2ffd700 1 -- 172.21.15.96:0/3257744449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe2bc09bb00 con 0x7fe2cc102ce0 2024-03-18T15:39:14.875 INFO:teuthology.orchestra.run.smithi096.stdout:292057776166 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 msgr2=0x7fe2b80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 0x7fe2b80a4de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8000c00 tx=0x7fe2c8005fb0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 msgr2=0x7fe2cc074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00e9a0 tx=0x7fe2bc00ecb0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 shutdown_connections 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2b80a2920 0x7fe2b80a4de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2cc108300 0x7fe2cc075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2cc102320 0x7fe2cc0745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 --2- 172.21.15.96:0/3257744449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2cc102ce0 0x7fe2cc074b10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 >> 172.21.15.96:0/3257744449 conn(0x7fe2cc079400 msgr2=0x7fe2cc100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:14.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 shutdown_connections 2024-03-18T15:39:14.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.876+0000 7fe2d424f700 1 -- 172.21.15.96:0/3257744449 wait complete. 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 -- 172.21.15.96:0/3501611133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101e10 msgr2=0x7f027c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 --2- 172.21.15.96:0/3501611133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101e10 0x7f027c105dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f02700099c0 tx=0x7f0270009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 -- 172.21.15.96:0/3501611133 shutdown_connections 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 --2- 172.21.15.96:0/3501611133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c10e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 --2- 172.21.15.96:0/3501611133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101460 0x7f027c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 --2- 172.21.15.96:0/3501611133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101e10 0x7f027c105dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 -- 172.21.15.96:0/3501611133 >> 172.21.15.96:0/3501611133 conn(0x7f027c077220 msgr2=0x7f027c077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:14.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 -- 172.21.15.96:0/3501611133 shutdown_connections 2024-03-18T15:39:14.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.891+0000 7f02837e8700 1 -- 172.21.15.96:0/3501611133 wait complete. 2024-03-18T15:39:14.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 Processor -- start 2024-03-18T15:39:14.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 -- start start 2024-03-18T15:39:14.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101e10 0x7f027c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f027c111470 con 0x7f027c101460 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1112f0 con 0x7f027c1064b0 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f02837e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1115f0 con 0x7f027c101e10 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f0281584700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f0281584700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56516/0 (socket says 172.21.15.96:56516) 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f0281584700 1 -- 172.21.15.96:0/611853482 learned_addr learned my addr 172.21.15.96:0/611853482 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f0281d85700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.892+0000 7f0280d83700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101e10 0x7f027c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 -- 172.21.15.96:0/611853482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101e10 msgr2=0x7f027c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101e10 0x7f027c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 -- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 msgr2=0x7f027c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 -- 172.21.15.96:0/611853482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c198030 con 0x7f027c1064b0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281584700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f0281d85700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c197750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f027400e970 tx=0x7f027400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274004220 con 0x7f027c1064b0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c198320 con 0x7f027c1064b0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0274004d10 con 0x7f027c1064b0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274010700 con 0x7f027c1064b0 2024-03-18T15:39:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.893+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a4600 con 0x7f027c1064b0 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.894+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027c1028b0 con 0x7f027c1064b0 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.895+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f0274004750 con 0x7f027c1064b0 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.896+0000 7f027a7fc700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 0x7f02640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.896+0000 7f0281584700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 0x7f02640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.896+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f02740ce880 con 0x7f027c1064b0 2024-03-18T15:39:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.896+0000 7f0281584700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 0x7f02640a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f026c009c10 tx=0x7f026c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:14.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:14.904+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f027401fe40 con 0x7f027c1064b0 2024-03-18T15:39:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:14 smithi193 ceph-mon[36702]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-18T15:39:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:14 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2436973276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-18T15:39:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.108+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f027c198c80 con 0x7f027c1064b0 2024-03-18T15:39:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.109+0000 7f027a7fc700 1 -- 172.21.15.96:0/611853482 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f027409b7b0 con 0x7f027c1064b0 2024-03-18T15:39:15.110 INFO:teuthology.orchestra.run.smithi096.stdout:167503724603 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 msgr2=0x7f02640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 0x7f02640a4de0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f026c009c10 tx=0x7f026c00b040 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 msgr2=0x7f027c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c197750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f027400e970 tx=0x7f027400ec80 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 shutdown_connections 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02640a2920 0x7f02640a4de0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f027c1064b0 0x7f027c197750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f027c101460 0x7f027c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 --2- 172.21.15.96:0/611853482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f027c101e10 0x7f027c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 >> 172.21.15.96:0/611853482 conn(0x7f027c077220 msgr2=0x7f027c0ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 shutdown_connections 2024-03-18T15:39:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:15.111+0000 7f02837e8700 1 -- 172.21.15.96:0/611853482 wait complete. 2024-03-18T15:39:15.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462965 for osd.6 2024-03-18T15:39:15.616 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:15.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:15 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3257744449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T15:39:15.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:15 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/611853482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T15:39:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:15 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3257744449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T15:39:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:15 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/611853482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T15:39:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:15 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3257744449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-18T15:39:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:15 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/611853482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-18T15:39:16.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724597 got 167503724603 for osd.5 2024-03-18T15:39:16.431 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:16.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:16 smithi096 ceph-mon[31930]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:16.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612634 got 313532612639 for osd.10 2024-03-18T15:39:16.802 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:16 smithi137 ceph-mon[35868]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:16.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:16 smithi193 ceph-mon[36702]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:16.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 -- 172.21.15.96:0/684691108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28106160 msgr2=0x7fad28106540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:16.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 --2- 172.21.15.96:0/684691108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28106160 0x7fad28106540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009990 tx=0x7fad1c009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 -- 172.21.15.96:0/684691108 shutdown_connections 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 --2- 172.21.15.96:0/684691108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28101ee0 0x7fad2810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 --2- 172.21.15.96:0/684691108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28106160 0x7fad28106540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 --2- 172.21.15.96:0/684691108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101520 0x7fad281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 -- 172.21.15.96:0/684691108 >> 172.21.15.96:0/684691108 conn(0x7fad28077f80 msgr2=0x7fad28078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 -- 172.21.15.96:0/684691108 shutdown_connections 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.985+0000 7fad2df80700 1 -- 172.21.15.96:0/684691108 wait complete. 2024-03-18T15:39:16.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 Processor -- start 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 -- start start 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101ee0 0x7fad2819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28106160 0x7fad281988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fad28111560 con 0x7fad28101520 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad281113e0 con 0x7fad28101ee0 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad2df80700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad281116e0 con 0x7fad28106160 2024-03-18T15:39:16.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56544/0 (socket says 172.21.15.96:56544) 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad277fe700 1 -- 172.21.15.96:0/3240142604 learned_addr learned my addr 172.21.15.96:0/3240142604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.986+0000 7fad26ffd700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101ee0 0x7fad2819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad27fff700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28106160 0x7fad281988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 -- 172.21.15.96:0/3240142604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28106160 msgr2=0x7fad281988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28106160 0x7fad281988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 -- 172.21.15.96:0/3240142604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101ee0 msgr2=0x7fad2819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101ee0 0x7fad2819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 -- 172.21.15.96:0/3240142604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28199030 con 0x7fad28101520 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad277fe700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fad1c000c00 tx=0x7fad1c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:16.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c027070 con 0x7fad28101520 2024-03-18T15:39:16.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad1c003d70 con 0x7fad28101520 2024-03-18T15:39:16.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad281992c0 con 0x7fad28101520 2024-03-18T15:39:16.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00f460 con 0x7fad28101520 2024-03-18T15:39:16.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.987+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281a5590 con 0x7fad28101520 2024-03-18T15:39:16.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.988+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad281028c0 con 0x7fad28101520 2024-03-18T15:39:16.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.989+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fad1c02a070 con 0x7fad28101520 2024-03-18T15:39:16.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.990+0000 7fad24ff9700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 0x7fad100a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:16.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.990+0000 7fad26ffd700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 0x7fad100a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:16.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.990+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fad1c0cf980 con 0x7fad28101520 2024-03-18T15:39:16.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.990+0000 7fad26ffd700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 0x7fad100a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad280785f0 tx=0x7fad18009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:16.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:16.994+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fad1c013070 con 0x7fad28101520 2024-03-18T15:39:17.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.198+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fad28199db0 con 0x7fad28101520 2024-03-18T15:39:17.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.198+0000 7fad24ff9700 1 -- 172.21.15.96:0/3240142604 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fad1c09c940 con 0x7fad28101520 2024-03-18T15:39:17.199 INFO:teuthology.orchestra.run.smithi096.stdout:347892351004 2024-03-18T15:39:17.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 msgr2=0x7fad100a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 0x7fad100a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad280785f0 tx=0x7fad18009260 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 msgr2=0x7fad2819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fad1c000c00 tx=0x7fad1c00bab0 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 shutdown_connections 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fad100a2920 0x7fad100a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad28106160 0x7fad281988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad28101520 0x7fad2819e6c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 --2- 172.21.15.96:0/3240142604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad28101ee0 0x7fad2819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.200+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 >> 172.21.15.96:0/3240142604 conn(0x7fad28077f80 msgr2=0x7fad2806a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.201+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 shutdown_connections 2024-03-18T15:39:17.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:17.201+0000 7fad2df80700 1 -- 172.21.15.96:0/3240142604 wait complete. 2024-03-18T15:39:17.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411400 for osd.2 2024-03-18T15:39:17.443 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:17.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:17 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3240142604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T15:39:17.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738450 for osd.0 2024-03-18T15:39:17.844 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:17.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:17 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3240142604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T15:39:17.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:17 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3240142604' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-18T15:39:18.498 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776166 for osd.9 2024-03-18T15:39:18.499 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:18.843 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:18 smithi096 ceph-mon[31930]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:18 smithi137 ceph-mon[35868]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:18.870 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351004 for osd.11 2024-03-18T15:39:18.870 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:18 smithi193 ceph-mon[36702]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-03-18T15:39:19.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574926 for osd.1 2024-03-18T15:39:19.249 DEBUG:teuthology.parallel:result is None 2024-03-18T15:39:19.249 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-18T15:39:19.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T15:39:20.170 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:20.597 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:20 smithi096 ceph-mon[31930]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:20 smithi137 ceph-mon[35868]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:20 smithi193 ceph-mon[36702]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:21.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 -- 172.21.15.96:0/2542298189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98741073c0 msgr2=0x7f98741077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:21.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 --2- 172.21.15.96:0/2542298189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98741073c0 0x7f98741077a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9868009990 tx=0x7f9868009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 -- 172.21.15.96:0/2542298189 shutdown_connections 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 --2- 172.21.15.96:0/2542298189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9874101e70 0x7f987410e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 --2- 172.21.15.96:0/2542298189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98741073c0 0x7f98741077a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 --2- 172.21.15.96:0/2542298189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741014b0 0x7f9874101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 -- 172.21.15.96:0/2542298189 >> 172.21.15.96:0/2542298189 conn(0x7f9874078030 msgr2=0x7f9874078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 -- 172.21.15.96:0/2542298189 shutdown_connections 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.937+0000 7f987adb8700 1 -- 172.21.15.96:0/2542298189 wait complete. 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 Processor -- start 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 -- start start 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 0x7f98741a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741073c0 0x7f987419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9874112650 con 0x7f9874101e70 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98741124d0 con 0x7f98741073c0 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f987adb8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98741127d0 con 0x7f98741014b0 2024-03-18T15:39:21.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f9873fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f9878b54700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 0x7f98741a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f9873fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56564/0 (socket says 172.21.15.96:56564) 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f9873fff700 1 -- 172.21.15.96:0/1892179306 learned_addr learned my addr 172.21.15.96:0/1892179306 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.938+0000 7f9879355700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741073c0 0x7f987419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 -- 172.21.15.96:0/1892179306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741073c0 msgr2=0x7f987419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741073c0 0x7f987419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 -- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 msgr2=0x7f98741a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 -- 172.21.15.96:0/1892179306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987419b210 con 0x7f98741014b0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9873fff700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9878b54700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 0x7f98741a08f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98680059d0 tx=0x7f986800bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868027070 con 0x7f98741014b0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987419b4a0 con 0x7f98741014b0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9868003d70 con 0x7f98741014b0 2024-03-18T15:39:21.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868022e30 con 0x7f98741014b0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.939+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a77c0 con 0x7f98741014b0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.940+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874102950 con 0x7f98741014b0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.941+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f986802a070 con 0x7f98741014b0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.942+0000 7f9871ffb700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 0x7f98640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.942+0000 7f9873fff700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 0x7f98640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.942+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f98680cfe70 con 0x7f98741014b0 2024-03-18T15:39:21.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.942+0000 7f9873fff700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 0x7f98640a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98740788d0 tx=0x7f9860009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:21.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:21.950+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9868013070 con 0x7f98741014b0 2024-03-18T15:39:22.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.162+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f987410ddb0 con 0x7f98640a2920 2024-03-18T15:39:22.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.171+0000 7f9871ffb700 1 -- 172.21.15.96:0/1892179306 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259031 (secure 0 0 0) 0x7f987410ddb0 con 0x7f98640a2920 2024-03-18T15:39:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:22.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 msgr2=0x7f98640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:22.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 0x7f98640a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98740788d0 tx=0x7f9860009260 comp rx=0 tx=0).stop 2024-03-18T15:39:22.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 msgr2=0x7f98741a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 0x7f98741a08f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98680059d0 tx=0x7f986800bab0 comp rx=0 tx=0).stop 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 shutdown_connections 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f98640a2920 0x7f98640a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98741073c0 0x7f987419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741014b0 0x7f98741a08f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 --2- 172.21.15.96:0/1892179306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9874101e70 0x7f98741a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 >> 172.21.15.96:0/1892179306 conn(0x7f9874078030 msgr2=0x7f987410c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 shutdown_connections 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:22.174+0000 7f987adb8700 1 -- 172.21.15.96:0/1892179306 wait complete. 2024-03-18T15:39:22.178 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:39:22.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:22 smithi096 ceph-mon[31930]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:22.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:22.827 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:22 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:22.853 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-03-18T15:39:21.461486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658070,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":848,"num_read_kb":547,"num_write":418,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365228,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576596,"statfs":{"total":1151940427776,"available":1151566434304,"internally_reserved":0,"allocated":12128256,"data_stored":7497042,"data_compressed":182655,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565612+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:39:02.565612+0000","last_peered":"2024-03-18T15:39:02.565612+0000","last_clean":"2024-03-18T15:39:02.565612+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:39:02.565612+0000","last_undegraded":"2024-03-18T15:39:02.565612+0000","last_fullsized":"2024-03-18T15:39:02.565612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569385+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:39:02.569385+0000","last_peered":"2024-03-18T15:39:02.569385+0000","last_clean":"2024-03-18T15:39:02.569385+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:39:02.569385+0000","last_undegraded":"2024-03-18T15:39:02.569385+0000","last_fullsized":"2024-03-18T15:39:02.569385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568934+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:39:02.568934+0000","last_peered":"2024-03-18T15:39:02.568934+0000","last_clean":"2024-03-18T15:39:02.568934+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:39:02.568934+0000","last_undegraded":"2024-03-18T15:39:02.568934+0000","last_fullsized":"2024-03-18T15:39:02.568934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568216+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:39:02.568216+0000","last_peered":"2024-03-18T15:39:02.568216+0000","last_clean":"2024-03-18T15:39:02.568216+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:39:02.568216+0000","last_undegraded":"2024-03-18T15:39:02.568216+0000","last_fullsized":"2024-03-18T15:39:02.568216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568397+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:39:02.568397+0000","last_peered":"2024-03-18T15:39:02.568397+0000","last_clean":"2024-03-18T15:39:02.568397+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:39:02.568397+0000","last_undegraded":"2024-03-18T15:39:02.568397+0000","last_fullsized":"2024-03-18T15:39:02.568397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609602+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:39:02.609602+0000","last_peered":"2024-03-18T15:39:02.609602+0000","last_clean":"2024-03-18T15:39:02.609602+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:39:02.609602+0000","last_undegraded":"2024-03-18T15:39:02.609602+0000","last_fullsized":"2024-03-18T15:39:02.609602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567824+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:39:02.567824+0000","last_peered":"2024-03-18T15:39:02.567824+0000","last_clean":"2024-03-18T15:39:02.567824+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:39:02.567824+0000","last_undegraded":"2024-03-18T15:39:02.567824+0000","last_fullsized":"2024-03-18T15:39:02.567824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569024+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:39:02.569024+0000","last_peered":"2024-03-18T15:39:02.569024+0000","last_clean":"2024-03-18T15:39:02.569024+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:39:02.569024+0000","last_undegraded":"2024-03-18T15:39:02.569024+0000","last_fullsized":"2024-03-18T15:39:02.569024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609801+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:39:02.609801+0000","last_peered":"2024-03-18T15:39:02.609801+0000","last_clean":"2024-03-18T15:39:02.609801+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:39:02.609801+0000","last_undegraded":"2024-03-18T15:39:02.609801+0000","last_fullsized":"2024-03-18T15:39:02.609801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610098+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:39:02.610098+0000","last_peered":"2024-03-18T15:39:02.610098+0000","last_clean":"2024-03-18T15:39:02.610098+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:39:02.610098+0000","last_undegraded":"2024-03-18T15:39:02.610098+0000","last_fullsized":"2024-03-18T15:39:02.610098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568396+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:39:02.568396+0000","last_peered":"2024-03-18T15:39:02.568396+0000","last_clean":"2024-03-18T15:39:02.568396+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:39:02.568396+0000","last_undegraded":"2024-03-18T15:39:02.568396+0000","last_fullsized":"2024-03-18T15:39:02.568396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568173+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:39:02.568173+0000","last_peered":"2024-03-18T15:39:02.568173+0000","last_clean":"2024-03-18T15:39:02.568173+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:39:02.568173+0000","last_undegraded":"2024-03-18T15:39:02.568173+0000","last_fullsized":"2024-03-18T15:39:02.568173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610008+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:39:02.610008+0000","last_peered":"2024-03-18T15:39:02.610008+0000","last_clean":"2024-03-18T15:39:02.610008+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:39:02.610008+0000","last_undegraded":"2024-03-18T15:39:02.610008+0000","last_fullsized":"2024-03-18T15:39:02.610008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569109+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:39:02.569109+0000","last_peered":"2024-03-18T15:39:02.569109+0000","last_clean":"2024-03-18T15:39:02.569109+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:39:02.569109+0000","last_undegraded":"2024-03-18T15:39:02.569109+0000","last_fullsized":"2024-03-18T15:39:02.569109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568824+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:39:02.568824+0000","last_peered":"2024-03-18T15:39:02.568824+0000","last_clean":"2024-03-18T15:39:02.568824+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:39:02.568824+0000","last_undegraded":"2024-03-18T15:39:02.568824+0000","last_fullsized":"2024-03-18T15:39:02.568824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609096+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:39:02.609096+0000","last_peered":"2024-03-18T15:39:02.609096+0000","last_clean":"2024-03-18T15:39:02.609096+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:39:02.609096+0000","last_undegraded":"2024-03-18T15:39:02.609096+0000","last_fullsized":"2024-03-18T15:39:02.609096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565509+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:39:02.565509+0000","last_peered":"2024-03-18T15:39:02.565509+0000","last_clean":"2024-03-18T15:39:02.565509+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:39:02.565509+0000","last_undegraded":"2024-03-18T15:39:02.565509+0000","last_fullsized":"2024-03-18T15:39:02.565509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565997+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:39:02.565997+0000","last_peered":"2024-03-18T15:39:02.565997+0000","last_clean":"2024-03-18T15:39:02.565997+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:39:02.565997+0000","last_undegraded":"2024-03-18T15:39:02.565997+0000","last_fullsized":"2024-03-18T15:39:02.565997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571298+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:39:02.571298+0000","last_peered":"2024-03-18T15:39:02.571298+0000","last_clean":"2024-03-18T15:39:02.571298+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:39:02.571298+0000","last_undegraded":"2024-03-18T15:39:02.571298+0000","last_fullsized":"2024-03-18T15:39:02.571298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:18.221939+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:39:18.221939+0000","last_peered":"2024-03-18T15:39:18.221939+0000","last_clean":"2024-03-18T15:39:18.221939+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:39:18.221939+0000","last_undegraded":"2024-03-18T15:39:18.221939+0000","last_fullsized":"2024-03-18T15:39:18.221939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.573598+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:39:02.573598+0000","last_peered":"2024-03-18T15:39:02.573598+0000","last_clean":"2024-03-18T15:39:02.573598+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:39:02.573598+0000","last_undegraded":"2024-03-18T15:39:02.573598+0000","last_fullsized":"2024-03-18T15:39:02.573598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570206+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:39:02.570206+0000","last_peered":"2024-03-18T15:39:02.570206+0000","last_clean":"2024-03-18T15:39:02.570206+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:39:02.570206+0000","last_undegraded":"2024-03-18T15:39:02.570206+0000","last_fullsized":"2024-03-18T15:39:02.570206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570073+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:39:02.570073+0000","last_peered":"2024-03-18T15:39:02.570073+0000","last_clean":"2024-03-18T15:39:02.570073+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:39:02.570073+0000","last_undegraded":"2024-03-18T15:39:02.570073+0000","last_fullsized":"2024-03-18T15:39:02.570073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569097+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:39:02.569097+0000","last_peered":"2024-03-18T15:39:02.569097+0000","last_clean":"2024-03-18T15:39:02.569097+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:39:02.569097+0000","last_undegraded":"2024-03-18T15:39:02.569097+0000","last_fullsized":"2024-03-18T15:39:02.569097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609356+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:39:02.609356+0000","last_peered":"2024-03-18T15:39:02.609356+0000","last_clean":"2024-03-18T15:39:02.609356+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:39:02.609356+0000","last_undegraded":"2024-03-18T15:39:02.609356+0000","last_fullsized":"2024-03-18T15:39:02.609356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568722+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:39:02.568722+0000","last_peered":"2024-03-18T15:39:02.568722+0000","last_clean":"2024-03-18T15:39:02.568722+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:39:02.568722+0000","last_undegraded":"2024-03-18T15:39:02.568722+0000","last_fullsized":"2024-03-18T15:39:02.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610855+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:39:02.610855+0000","last_peered":"2024-03-18T15:39:02.610855+0000","last_clean":"2024-03-18T15:39:02.610855+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:39:02.610855+0000","last_undegraded":"2024-03-18T15:39:02.610855+0000","last_fullsized":"2024-03-18T15:39:02.610855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:16.218938+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:39:16.218938+0000","last_peered":"2024-03-18T15:39:16.218938+0000","last_clean":"2024-03-18T15:39:16.218938+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:39:16.218938+0000","last_undegraded":"2024-03-18T15:39:16.218938+0000","last_fullsized":"2024-03-18T15:39:16.218938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568391+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:39:02.568391+0000","last_peered":"2024-03-18T15:39:02.568391+0000","last_clean":"2024-03-18T15:39:02.568391+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:39:02.568391+0000","last_undegraded":"2024-03-18T15:39:02.568391+0000","last_fullsized":"2024-03-18T15:39:02.568391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609466+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:39:02.609466+0000","last_peered":"2024-03-18T15:39:02.609466+0000","last_clean":"2024-03-18T15:39:02.609466+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:39:02.609466+0000","last_undegraded":"2024-03-18T15:39:02.609466+0000","last_fullsized":"2024-03-18T15:39:02.609466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568601+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:39:02.568601+0000","last_peered":"2024-03-18T15:39:02.568601+0000","last_clean":"2024-03-18T15:39:02.568601+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:39:02.568601+0000","last_undegraded":"2024-03-18T15:39:02.568601+0000","last_fullsized":"2024-03-18T15:39:02.568601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609673+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:39:02.609673+0000","last_peered":"2024-03-18T15:39:02.609673+0000","last_clean":"2024-03-18T15:39:02.609673+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:39:02.609673+0000","last_undegraded":"2024-03-18T15:39:02.609673+0000","last_fullsized":"2024-03-18T15:39:02.609673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569145+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:39:02.569145+0000","last_peered":"2024-03-18T15:39:02.569145+0000","last_clean":"2024-03-18T15:39:02.569145+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:39:02.569145+0000","last_undegraded":"2024-03-18T15:39:02.569145+0000","last_fullsized":"2024-03-18T15:39:02.569145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568251+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:39:02.568251+0000","last_peered":"2024-03-18T15:39:02.568251+0000","last_clean":"2024-03-18T15:39:02.568251+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:39:02.568251+0000","last_undegraded":"2024-03-18T15:39:02.568251+0000","last_fullsized":"2024-03-18T15:39:02.568251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565656+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:39:02.565656+0000","last_peered":"2024-03-18T15:39:02.565656+0000","last_clean":"2024-03-18T15:39:02.565656+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:39:02.565656+0000","last_undegraded":"2024-03-18T15:39:02.565656+0000","last_fullsized":"2024-03-18T15:39:02.565656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":163,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:20.222395+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:39:20.222395+0000","last_peered":"2024-03-18T15:39:20.222395+0000","last_clean":"2024-03-18T15:39:20.222395+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:39:20.222395+0000","last_undegraded":"2024-03-18T15:39:20.222395+0000","last_fullsized":"2024-03-18T15:39:20.222395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609699+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:39:02.609699+0000","last_peered":"2024-03-18T15:39:02.609699+0000","last_clean":"2024-03-18T15:39:02.609699+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:39:02.609699+0000","last_undegraded":"2024-03-18T15:39:02.609699+0000","last_fullsized":"2024-03-18T15:39:02.609699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568499+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:39:02.568499+0000","last_peered":"2024-03-18T15:39:02.568499+0000","last_clean":"2024-03-18T15:39:02.568499+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:39:02.568499+0000","last_undegraded":"2024-03-18T15:39:02.568499+0000","last_fullsized":"2024-03-18T15:39:02.568499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.611014+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:39:02.611014+0000","last_peered":"2024-03-18T15:39:02.611014+0000","last_clean":"2024-03-18T15:39:02.611014+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:39:02.611014+0000","last_undegraded":"2024-03-18T15:39:02.611014+0000","last_fullsized":"2024-03-18T15:39:02.611014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569009+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:39:02.569009+0000","last_peered":"2024-03-18T15:39:02.569009+0000","last_clean":"2024-03-18T15:39:02.569009+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:39:02.569009+0000","last_undegraded":"2024-03-18T15:39:02.569009+0000","last_fullsized":"2024-03-18T15:39:02.569009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609527+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:39:02.609527+0000","last_peered":"2024-03-18T15:39:02.609527+0000","last_clean":"2024-03-18T15:39:02.609527+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:39:02.609527+0000","last_undegraded":"2024-03-18T15:39:02.609527+0000","last_fullsized":"2024-03-18T15:39:02.609527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":195,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:20.222380+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:39:20.222380+0000","last_peered":"2024-03-18T15:39:20.222380+0000","last_clean":"2024-03-18T15:39:20.222380+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:39:20.222380+0000","last_undegraded":"2024-03-18T15:39:20.222380+0000","last_fullsized":"2024-03-18T15:39:20.222380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":133,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568902+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:39:02.568902+0000","last_peered":"2024-03-18T15:39:02.568902+0000","last_clean":"2024-03-18T15:39:02.568902+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:39:02.568902+0000","last_undegraded":"2024-03-18T15:39:02.568902+0000","last_fullsized":"2024-03-18T15:39:02.568902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571905+0000","last_change":"2024-03-18T15:38:03.834819+0000","last_active":"2024-03-18T15:39:02.571905+0000","last_peered":"2024-03-18T15:39:02.571905+0000","last_clean":"2024-03-18T15:39:02.571905+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:39:02.571905+0000","last_undegraded":"2024-03-18T15:39:02.571905+0000","last_fullsized":"2024-03-18T15:39:02.571905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:02:11.448328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609821+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:39:02.609821+0000","last_peered":"2024-03-18T15:39:02.609821+0000","last_clean":"2024-03-18T15:39:02.609821+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:39:02.609821+0000","last_undegraded":"2024-03-18T15:39:02.609821+0000","last_fullsized":"2024-03-18T15:39:02.609821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568342+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:39:02.568342+0000","last_peered":"2024-03-18T15:39:02.568342+0000","last_clean":"2024-03-18T15:39:02.568342+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:39:02.568342+0000","last_undegraded":"2024-03-18T15:39:02.568342+0000","last_fullsized":"2024-03-18T15:39:02.568342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568642+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:39:02.568642+0000","last_peered":"2024-03-18T15:39:02.568642+0000","last_clean":"2024-03-18T15:39:02.568642+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:39:02.568642+0000","last_undegraded":"2024-03-18T15:39:02.568642+0000","last_fullsized":"2024-03-18T15:39:02.568642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:17.221083+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:39:17.221083+0000","last_peered":"2024-03-18T15:39:17.221083+0000","last_clean":"2024-03-18T15:39:17.221083+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:39:17.221083+0000","last_undegraded":"2024-03-18T15:39:17.221083+0000","last_fullsized":"2024-03-18T15:39:17.221083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567566+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:39:02.567566+0000","last_peered":"2024-03-18T15:39:02.567566+0000","last_clean":"2024-03-18T15:39:02.567566+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:39:02.567566+0000","last_undegraded":"2024-03-18T15:39:02.567566+0000","last_fullsized":"2024-03-18T15:39:02.567566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567940+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:39:02.567940+0000","last_peered":"2024-03-18T15:39:02.567940+0000","last_clean":"2024-03-18T15:39:02.567940+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:39:02.567940+0000","last_undegraded":"2024-03-18T15:39:02.567940+0000","last_fullsized":"2024-03-18T15:39:02.567940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567932+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:39:02.567932+0000","last_peered":"2024-03-18T15:39:02.567932+0000","last_clean":"2024-03-18T15:39:02.567932+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:39:02.567932+0000","last_undegraded":"2024-03-18T15:39:02.567932+0000","last_fullsized":"2024-03-18T15:39:02.567932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.578097+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:39:02.578097+0000","last_peered":"2024-03-18T15:39:02.578097+0000","last_clean":"2024-03-18T15:39:02.578097+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:39:02.578097+0000","last_undegraded":"2024-03-18T15:39:02.578097+0000","last_fullsized":"2024-03-18T15:39:02.578097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571921+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:39:02.571921+0000","last_peered":"2024-03-18T15:39:02.571921+0000","last_clean":"2024-03-18T15:39:02.571921+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:39:02.571921+0000","last_undegraded":"2024-03-18T15:39:02.571921+0000","last_fullsized":"2024-03-18T15:39:02.571921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565718+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:39:02.565718+0000","last_peered":"2024-03-18T15:39:02.565718+0000","last_clean":"2024-03-18T15:39:02.565718+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:39:02.565718+0000","last_undegraded":"2024-03-18T15:39:02.565718+0000","last_fullsized":"2024-03-18T15:39:02.565718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609902+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:39:02.609902+0000","last_peered":"2024-03-18T15:39:02.609902+0000","last_clean":"2024-03-18T15:39:02.609902+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:39:02.609902+0000","last_undegraded":"2024-03-18T15:39:02.609902+0000","last_fullsized":"2024-03-18T15:39:02.609902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:17.221139+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:39:17.221139+0000","last_peered":"2024-03-18T15:39:17.221139+0000","last_clean":"2024-03-18T15:39:17.221139+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:39:17.221139+0000","last_undegraded":"2024-03-18T15:39:17.221139+0000","last_fullsized":"2024-03-18T15:39:17.221139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609806+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:39:02.609806+0000","last_peered":"2024-03-18T15:39:02.609806+0000","last_clean":"2024-03-18T15:39:02.609806+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:39:02.609806+0000","last_undegraded":"2024-03-18T15:39:02.609806+0000","last_fullsized":"2024-03-18T15:39:02.609806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568433+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:39:02.568433+0000","last_peered":"2024-03-18T15:39:02.568433+0000","last_clean":"2024-03-18T15:39:02.568433+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:39:02.568433+0000","last_undegraded":"2024-03-18T15:39:02.568433+0000","last_fullsized":"2024-03-18T15:39:02.568433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:16.219718+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:39:16.219718+0000","last_peered":"2024-03-18T15:39:16.219718+0000","last_clean":"2024-03-18T15:39:16.219718+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:39:16.219718+0000","last_undegraded":"2024-03-18T15:39:16.219718+0000","last_fullsized":"2024-03-18T15:39:16.219718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610861+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:39:02.610861+0000","last_peered":"2024-03-18T15:39:02.610861+0000","last_clean":"2024-03-18T15:39:02.610861+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:39:02.610861+0000","last_undegraded":"2024-03-18T15:39:02.610861+0000","last_fullsized":"2024-03-18T15:39:02.610861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567818+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:39:02.567818+0000","last_peered":"2024-03-18T15:39:02.567818+0000","last_clean":"2024-03-18T15:39:02.567818+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:39:02.567818+0000","last_undegraded":"2024-03-18T15:39:02.567818+0000","last_fullsized":"2024-03-18T15:39:02.567818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567855+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:39:02.567855+0000","last_peered":"2024-03-18T15:39:02.567855+0000","last_clean":"2024-03-18T15:39:02.567855+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:39:02.567855+0000","last_undegraded":"2024-03-18T15:39:02.567855+0000","last_fullsized":"2024-03-18T15:39:02.567855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609547+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:39:02.609547+0000","last_peered":"2024-03-18T15:39:02.609547+0000","last_clean":"2024-03-18T15:39:02.609547+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:39:02.609547+0000","last_undegraded":"2024-03-18T15:39:02.609547+0000","last_fullsized":"2024-03-18T15:39:02.609547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567978+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:39:02.567978+0000","last_peered":"2024-03-18T15:39:02.567978+0000","last_clean":"2024-03-18T15:39:02.567978+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:39:02.567978+0000","last_undegraded":"2024-03-18T15:39:02.567978+0000","last_fullsized":"2024-03-18T15:39:02.567978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568992+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:39:02.568992+0000","last_peered":"2024-03-18T15:39:02.568992+0000","last_clean":"2024-03-18T15:39:02.568992+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:39:02.568992+0000","last_undegraded":"2024-03-18T15:39:02.568992+0000","last_fullsized":"2024-03-18T15:39:02.568992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609980+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:39:02.609980+0000","last_peered":"2024-03-18T15:39:02.609980+0000","last_clean":"2024-03-18T15:39:02.609980+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:39:02.609980+0000","last_undegraded":"2024-03-18T15:39:02.609980+0000","last_fullsized":"2024-03-18T15:39:02.609980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.566128+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:39:02.566128+0000","last_peered":"2024-03-18T15:39:02.566128+0000","last_clean":"2024-03-18T15:39:02.566128+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:39:02.566128+0000","last_undegraded":"2024-03-18T15:39:02.566128+0000","last_fullsized":"2024-03-18T15:39:02.566128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569239+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:39:02.569239+0000","last_peered":"2024-03-18T15:39:02.569239+0000","last_clean":"2024-03-18T15:39:02.569239+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:39:02.569239+0000","last_undegraded":"2024-03-18T15:39:02.569239+0000","last_fullsized":"2024-03-18T15:39:02.569239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568297+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:39:02.568297+0000","last_peered":"2024-03-18T15:39:02.568297+0000","last_clean":"2024-03-18T15:39:02.568297+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:39:02.568297+0000","last_undegraded":"2024-03-18T15:39:02.568297+0000","last_fullsized":"2024-03-18T15:39:02.568297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569289+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:39:02.569289+0000","last_peered":"2024-03-18T15:39:02.569289+0000","last_clean":"2024-03-18T15:39:02.569289+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:39:02.569289+0000","last_undegraded":"2024-03-18T15:39:02.569289+0000","last_fullsized":"2024-03-18T15:39:02.569289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609419+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:39:02.609419+0000","last_peered":"2024-03-18T15:39:02.609419+0000","last_clean":"2024-03-18T15:39:02.609419+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:39:02.609419+0000","last_undegraded":"2024-03-18T15:39:02.609419+0000","last_fullsized":"2024-03-18T15:39:02.609419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568062+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:39:02.568062+0000","last_peered":"2024-03-18T15:39:02.568062+0000","last_clean":"2024-03-18T15:39:02.568062+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:39:02.568062+0000","last_undegraded":"2024-03-18T15:39:02.568062+0000","last_fullsized":"2024-03-18T15:39:02.568062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569256+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:39:02.569256+0000","last_peered":"2024-03-18T15:39:02.569256+0000","last_clean":"2024-03-18T15:39:02.569256+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:39:02.569256+0000","last_undegraded":"2024-03-18T15:39:02.569256+0000","last_fullsized":"2024-03-18T15:39:02.569256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565066+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:39:02.565066+0000","last_peered":"2024-03-18T15:39:02.565066+0000","last_clean":"2024-03-18T15:39:02.565066+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:39:02.565066+0000","last_undegraded":"2024-03-18T15:39:02.565066+0000","last_fullsized":"2024-03-18T15:39:02.565066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570820+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:39:02.570820+0000","last_peered":"2024-03-18T15:39:02.570820+0000","last_clean":"2024-03-18T15:39:02.570820+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:39:02.570820+0000","last_undegraded":"2024-03-18T15:39:02.570820+0000","last_fullsized":"2024-03-18T15:39:02.570820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569730+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:39:02.569730+0000","last_peered":"2024-03-18T15:39:02.569730+0000","last_clean":"2024-03-18T15:39:02.569730+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:39:02.569730+0000","last_undegraded":"2024-03-18T15:39:02.569730+0000","last_fullsized":"2024-03-18T15:39:02.569730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.581338+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:39:02.581338+0000","last_peered":"2024-03-18T15:39:02.581338+0000","last_clean":"2024-03-18T15:39:02.581338+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:39:02.581338+0000","last_undegraded":"2024-03-18T15:39:02.581338+0000","last_fullsized":"2024-03-18T15:39:02.581338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568086+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:39:02.568086+0000","last_peered":"2024-03-18T15:39:02.568086+0000","last_clean":"2024-03-18T15:39:02.568086+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:39:02.568086+0000","last_undegraded":"2024-03-18T15:39:02.568086+0000","last_fullsized":"2024-03-18T15:39:02.568086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:17.232440+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:39:17.232440+0000","last_peered":"2024-03-18T15:39:17.232440+0000","last_clean":"2024-03-18T15:39:17.232440+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:39:17.232440+0000","last_undegraded":"2024-03-18T15:39:17.232440+0000","last_fullsized":"2024-03-18T15:39:17.232440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568443+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:39:02.568443+0000","last_peered":"2024-03-18T15:39:02.568443+0000","last_clean":"2024-03-18T15:39:02.568443+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:39:02.568443+0000","last_undegraded":"2024-03-18T15:39:02.568443+0000","last_fullsized":"2024-03-18T15:39:02.568443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568502+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:39:02.568502+0000","last_peered":"2024-03-18T15:39:02.568502+0000","last_clean":"2024-03-18T15:39:02.568502+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:39:02.568502+0000","last_undegraded":"2024-03-18T15:39:02.568502+0000","last_fullsized":"2024-03-18T15:39:02.568502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568235+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:39:02.568235+0000","last_peered":"2024-03-18T15:39:02.568235+0000","last_clean":"2024-03-18T15:39:02.568235+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:39:02.568235+0000","last_undegraded":"2024-03-18T15:39:02.568235+0000","last_fullsized":"2024-03-18T15:39:02.568235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609878+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:39:02.609878+0000","last_peered":"2024-03-18T15:39:02.609878+0000","last_clean":"2024-03-18T15:39:02.609878+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:39:02.609878+0000","last_undegraded":"2024-03-18T15:39:02.609878+0000","last_fullsized":"2024-03-18T15:39:02.609878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568293+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:39:02.568293+0000","last_peered":"2024-03-18T15:39:02.568293+0000","last_clean":"2024-03-18T15:39:02.568293+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:39:02.568293+0000","last_undegraded":"2024-03-18T15:39:02.568293+0000","last_fullsized":"2024-03-18T15:39:02.568293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565134+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:39:02.565134+0000","last_peered":"2024-03-18T15:39:02.565134+0000","last_clean":"2024-03-18T15:39:02.565134+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:39:02.565134+0000","last_undegraded":"2024-03-18T15:39:02.565134+0000","last_fullsized":"2024-03-18T15:39:02.565134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571006+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:39:02.571006+0000","last_peered":"2024-03-18T15:39:02.571006+0000","last_clean":"2024-03-18T15:39:02.571006+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:39:02.571006+0000","last_undegraded":"2024-03-18T15:39:02.571006+0000","last_fullsized":"2024-03-18T15:39:02.571006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.574194+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:39:02.574194+0000","last_peered":"2024-03-18T15:39:02.574194+0000","last_clean":"2024-03-18T15:39:02.574194+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:39:02.574194+0000","last_undegraded":"2024-03-18T15:39:02.574194+0000","last_fullsized":"2024-03-18T15:39:02.574194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569500+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:39:02.569500+0000","last_peered":"2024-03-18T15:39:02.569500+0000","last_clean":"2024-03-18T15:39:02.569500+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:39:02.569500+0000","last_undegraded":"2024-03-18T15:39:02.569500+0000","last_fullsized":"2024-03-18T15:39:02.569500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609768+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:39:02.609768+0000","last_peered":"2024-03-18T15:39:02.609768+0000","last_clean":"2024-03-18T15:39:02.609768+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:39:02.609768+0000","last_undegraded":"2024-03-18T15:39:02.609768+0000","last_fullsized":"2024-03-18T15:39:02.609768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565343+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:39:02.565343+0000","last_peered":"2024-03-18T15:39:02.565343+0000","last_clean":"2024-03-18T15:39:02.565343+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:39:02.565343+0000","last_undegraded":"2024-03-18T15:39:02.565343+0000","last_fullsized":"2024-03-18T15:39:02.565343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570696+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:39:02.570696+0000","last_peered":"2024-03-18T15:39:02.570696+0000","last_clean":"2024-03-18T15:39:02.570696+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:39:02.570696+0000","last_undegraded":"2024-03-18T15:39:02.570696+0000","last_fullsized":"2024-03-18T15:39:02.570696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569117+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:39:02.569117+0000","last_peered":"2024-03-18T15:39:02.569117+0000","last_clean":"2024-03-18T15:39:02.569117+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:39:02.569117+0000","last_undegraded":"2024-03-18T15:39:02.569117+0000","last_fullsized":"2024-03-18T15:39:02.569117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568348+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:39:02.568348+0000","last_peered":"2024-03-18T15:39:02.568348+0000","last_clean":"2024-03-18T15:39:02.568348+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:39:02.568348+0000","last_undegraded":"2024-03-18T15:39:02.568348+0000","last_fullsized":"2024-03-18T15:39:02.568348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569077+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:39:02.569077+0000","last_peered":"2024-03-18T15:39:02.569077+0000","last_clean":"2024-03-18T15:39:02.569077+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:39:02.569077+0000","last_undegraded":"2024-03-18T15:39:02.569077+0000","last_fullsized":"2024-03-18T15:39:02.569077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609471+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:39:02.609471+0000","last_peered":"2024-03-18T15:39:02.609471+0000","last_clean":"2024-03-18T15:39:02.609471+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:39:02.609471+0000","last_undegraded":"2024-03-18T15:39:02.609471+0000","last_fullsized":"2024-03-18T15:39:02.609471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610015+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:39:02.610015+0000","last_peered":"2024-03-18T15:39:02.610015+0000","last_clean":"2024-03-18T15:39:02.610015+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:39:02.610015+0000","last_undegraded":"2024-03-18T15:39:02.610015+0000","last_fullsized":"2024-03-18T15:39:02.610015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569338+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:39:02.569338+0000","last_peered":"2024-03-18T15:39:02.569338+0000","last_clean":"2024-03-18T15:39:02.569338+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:39:02.569338+0000","last_undegraded":"2024-03-18T15:39:02.569338+0000","last_fullsized":"2024-03-18T15:39:02.569338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67468,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":722,"num_read_kb":438,"num_write":235,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":202404,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":461450,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1113503,"data_compressed":43320,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461358,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":461714,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1053863,"data_compressed":41452,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":461075,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":523034,"data_compressed":7723,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":462414,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":523254,"data_compressed":7723,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460900,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460936,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1053541,"data_compressed":41452,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62782,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:39:22.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-18T15:39:22.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:22 smithi137 ceph-mon[35868]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:22.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:22.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:22 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:22 smithi193 ceph-mon[36702]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:22 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:23.103 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:23 smithi137 ceph-mon[35868]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:39:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:39:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:23 smithi193 ceph-mon[36702]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:39:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:39:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:23.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:23 smithi096 ceph-mon[31930]: from='client.24826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:23.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:39:23.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:39:23.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:39:24.770 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:24 smithi096 ceph-mon[31930]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 -- 172.21.15.96:0/3980789918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63041064b0 msgr2=0x7f630410e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 --2- 172.21.15.96:0/3980789918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63041064b0 0x7f630410e080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b0d0 tx=0x7f62fc00b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 -- 172.21.15.96:0/3980789918 shutdown_connections 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 --2- 172.21.15.96:0/3980789918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63041064b0 0x7f630410e080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 --2- 172.21.15.96:0/3980789918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 0x7f6304101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 --2- 172.21.15.96:0/3980789918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6304101e10 0x7f6304105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 -- 172.21.15.96:0/3980789918 >> 172.21.15.96:0/3980789918 conn(0x7f6304077220 msgr2=0x7f6304077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 -- 172.21.15.96:0/3980789918 shutdown_connections 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.836+0000 7f630953a700 1 -- 172.21.15.96:0/3980789918 wait complete. 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 Processor -- start 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 -- start start 2024-03-18T15:39:24.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 0x7f6304199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 0x7f6304199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 0x7f63041a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63041114c0 con 0x7f6304101e10 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6304111340 con 0x7f6304101460 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f630953a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6304111640 con 0x7f63041064b0 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 0x7f6304199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 0x7f63041a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63027fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 0x7f6304199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36512/0 (socket says 172.21.15.96:36512) 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63027fc700 1 -- 172.21.15.96:0/1199817609 learned_addr learned my addr 172.21.15.96:0/1199817609 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f6302ffd700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 0x7f6304199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 -- 172.21.15.96:0/1199817609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 msgr2=0x7f6304199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 0x7f6304199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 -- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 msgr2=0x7f6304199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 0x7f6304199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.837+0000 7f63037fe700 1 -- 172.21.15.96:0/1199817609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63041a4de0 con 0x7f63041064b0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f63037fe700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 0x7f63041a48a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b770 tx=0x7f62fc007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc022070 con 0x7f63041064b0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6304197610 con 0x7f63041064b0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62fc01cda0 con 0x7f63041064b0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc026bb0 con 0x7f63041064b0 2024-03-18T15:39:24.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.838+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6304197a70 con 0x7f63041064b0 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.839+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6304068f90 con 0x7f63041064b0 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.840+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f62fc009b80 con 0x7f63041064b0 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.841+0000 7f62ebfff700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 0x7f62ec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.841+0000 7f6302ffd700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 0x7f62ec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.841+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f62fc0d3690 con 0x7f63041064b0 2024-03-18T15:39:24.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.841+0000 7f6302ffd700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 0x7f62ec0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62f4007be0 tx=0x7f62f40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:24.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:24.844+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f62fc0a0700 con 0x7f63041064b0 2024-03-18T15:39:24.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:24 smithi137 ceph-mon[35868]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:24.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:24 smithi193 ceph-mon[36702]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.040+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6304062710 con 0x7f62ec0a2920 2024-03-18T15:39:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.048+0000 7f62ebfff700 1 -- 172.21.15.96:0/1199817609 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259031 (secure 0 0 0) 0x7f6304062710 con 0x7f62ec0a2920 2024-03-18T15:39:25.052 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 msgr2=0x7f62ec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 0x7f62ec0a4de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f62f4007be0 tx=0x7f62f40073d0 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 msgr2=0x7f63041a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 0x7f63041a48a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b770 tx=0x7f62fc007a20 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 shutdown_connections 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f62ec0a2920 0x7f62ec0a4de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63041064b0 0x7f63041a48a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6304101460 0x7f6304199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 --2- 172.21.15.96:0/1199817609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6304101e10 0x7f6304199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.051+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 >> 172.21.15.96:0/1199817609 conn(0x7f6304077220 msgr2=0x7f63040ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.052+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 shutdown_connections 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:25.052+0000 7f630953a700 1 -- 172.21.15.96:0/1199817609 wait complete. 2024-03-18T15:39:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:39:25.748 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-03-18T15:39:23.461933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658070,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":868,"num_read_kb":552,"num_write":418,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365228,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576596,"statfs":{"total":1151940427776,"available":1151566434304,"internally_reserved":0,"allocated":12128256,"data_stored":7497042,"data_compressed":182655,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002647"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565612+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:39:02.565612+0000","last_peered":"2024-03-18T15:39:02.565612+0000","last_clean":"2024-03-18T15:39:02.565612+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:39:02.565612+0000","last_undegraded":"2024-03-18T15:39:02.565612+0000","last_fullsized":"2024-03-18T15:39:02.565612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569385+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:39:02.569385+0000","last_peered":"2024-03-18T15:39:02.569385+0000","last_clean":"2024-03-18T15:39:02.569385+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:39:02.569385+0000","last_undegraded":"2024-03-18T15:39:02.569385+0000","last_fullsized":"2024-03-18T15:39:02.569385+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568934+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:39:02.568934+0000","last_peered":"2024-03-18T15:39:02.568934+0000","last_clean":"2024-03-18T15:39:02.568934+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:39:02.568934+0000","last_undegraded":"2024-03-18T15:39:02.568934+0000","last_fullsized":"2024-03-18T15:39:02.568934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568216+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:39:02.568216+0000","last_peered":"2024-03-18T15:39:02.568216+0000","last_clean":"2024-03-18T15:39:02.568216+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:39:02.568216+0000","last_undegraded":"2024-03-18T15:39:02.568216+0000","last_fullsized":"2024-03-18T15:39:02.568216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568397+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:39:02.568397+0000","last_peered":"2024-03-18T15:39:02.568397+0000","last_clean":"2024-03-18T15:39:02.568397+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:39:02.568397+0000","last_undegraded":"2024-03-18T15:39:02.568397+0000","last_fullsized":"2024-03-18T15:39:02.568397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609602+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:39:02.609602+0000","last_peered":"2024-03-18T15:39:02.609602+0000","last_clean":"2024-03-18T15:39:02.609602+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:39:02.609602+0000","last_undegraded":"2024-03-18T15:39:02.609602+0000","last_fullsized":"2024-03-18T15:39:02.609602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567824+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:39:02.567824+0000","last_peered":"2024-03-18T15:39:02.567824+0000","last_clean":"2024-03-18T15:39:02.567824+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:39:02.567824+0000","last_undegraded":"2024-03-18T15:39:02.567824+0000","last_fullsized":"2024-03-18T15:39:02.567824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569024+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:39:02.569024+0000","last_peered":"2024-03-18T15:39:02.569024+0000","last_clean":"2024-03-18T15:39:02.569024+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:39:02.569024+0000","last_undegraded":"2024-03-18T15:39:02.569024+0000","last_fullsized":"2024-03-18T15:39:02.569024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609801+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:39:02.609801+0000","last_peered":"2024-03-18T15:39:02.609801+0000","last_clean":"2024-03-18T15:39:02.609801+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:39:02.609801+0000","last_undegraded":"2024-03-18T15:39:02.609801+0000","last_fullsized":"2024-03-18T15:39:02.609801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610098+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:39:02.610098+0000","last_peered":"2024-03-18T15:39:02.610098+0000","last_clean":"2024-03-18T15:39:02.610098+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:39:02.610098+0000","last_undegraded":"2024-03-18T15:39:02.610098+0000","last_fullsized":"2024-03-18T15:39:02.610098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568396+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:39:02.568396+0000","last_peered":"2024-03-18T15:39:02.568396+0000","last_clean":"2024-03-18T15:39:02.568396+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:39:02.568396+0000","last_undegraded":"2024-03-18T15:39:02.568396+0000","last_fullsized":"2024-03-18T15:39:02.568396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568173+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:39:02.568173+0000","last_peered":"2024-03-18T15:39:02.568173+0000","last_clean":"2024-03-18T15:39:02.568173+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:39:02.568173+0000","last_undegraded":"2024-03-18T15:39:02.568173+0000","last_fullsized":"2024-03-18T15:39:02.568173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610008+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:39:02.610008+0000","last_peered":"2024-03-18T15:39:02.610008+0000","last_clean":"2024-03-18T15:39:02.610008+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:39:02.610008+0000","last_undegraded":"2024-03-18T15:39:02.610008+0000","last_fullsized":"2024-03-18T15:39:02.610008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569109+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:39:02.569109+0000","last_peered":"2024-03-18T15:39:02.569109+0000","last_clean":"2024-03-18T15:39:02.569109+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:39:02.569109+0000","last_undegraded":"2024-03-18T15:39:02.569109+0000","last_fullsized":"2024-03-18T15:39:02.569109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568824+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:39:02.568824+0000","last_peered":"2024-03-18T15:39:02.568824+0000","last_clean":"2024-03-18T15:39:02.568824+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:39:02.568824+0000","last_undegraded":"2024-03-18T15:39:02.568824+0000","last_fullsized":"2024-03-18T15:39:02.568824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609096+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:39:02.609096+0000","last_peered":"2024-03-18T15:39:02.609096+0000","last_clean":"2024-03-18T15:39:02.609096+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:39:02.609096+0000","last_undegraded":"2024-03-18T15:39:02.609096+0000","last_fullsized":"2024-03-18T15:39:02.609096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565509+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:39:02.565509+0000","last_peered":"2024-03-18T15:39:02.565509+0000","last_clean":"2024-03-18T15:39:02.565509+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:39:02.565509+0000","last_undegraded":"2024-03-18T15:39:02.565509+0000","last_fullsized":"2024-03-18T15:39:02.565509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565997+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:39:02.565997+0000","last_peered":"2024-03-18T15:39:02.565997+0000","last_clean":"2024-03-18T15:39:02.565997+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:39:02.565997+0000","last_undegraded":"2024-03-18T15:39:02.565997+0000","last_fullsized":"2024-03-18T15:39:02.565997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571298+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:39:02.571298+0000","last_peered":"2024-03-18T15:39:02.571298+0000","last_clean":"2024-03-18T15:39:02.571298+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:39:02.571298+0000","last_undegraded":"2024-03-18T15:39:02.571298+0000","last_fullsized":"2024-03-18T15:39:02.571298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:18.221939+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:39:18.221939+0000","last_peered":"2024-03-18T15:39:18.221939+0000","last_clean":"2024-03-18T15:39:18.221939+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:39:18.221939+0000","last_undegraded":"2024-03-18T15:39:18.221939+0000","last_fullsized":"2024-03-18T15:39:18.221939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.573598+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:39:02.573598+0000","last_peered":"2024-03-18T15:39:02.573598+0000","last_clean":"2024-03-18T15:39:02.573598+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:39:02.573598+0000","last_undegraded":"2024-03-18T15:39:02.573598+0000","last_fullsized":"2024-03-18T15:39:02.573598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570206+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:39:02.570206+0000","last_peered":"2024-03-18T15:39:02.570206+0000","last_clean":"2024-03-18T15:39:02.570206+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:39:02.570206+0000","last_undegraded":"2024-03-18T15:39:02.570206+0000","last_fullsized":"2024-03-18T15:39:02.570206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570073+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:39:02.570073+0000","last_peered":"2024-03-18T15:39:02.570073+0000","last_clean":"2024-03-18T15:39:02.570073+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:39:02.570073+0000","last_undegraded":"2024-03-18T15:39:02.570073+0000","last_fullsized":"2024-03-18T15:39:02.570073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569097+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:39:02.569097+0000","last_peered":"2024-03-18T15:39:02.569097+0000","last_clean":"2024-03-18T15:39:02.569097+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:39:02.569097+0000","last_undegraded":"2024-03-18T15:39:02.569097+0000","last_fullsized":"2024-03-18T15:39:02.569097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609356+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:39:02.609356+0000","last_peered":"2024-03-18T15:39:02.609356+0000","last_clean":"2024-03-18T15:39:02.609356+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:39:02.609356+0000","last_undegraded":"2024-03-18T15:39:02.609356+0000","last_fullsized":"2024-03-18T15:39:02.609356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568722+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:39:02.568722+0000","last_peered":"2024-03-18T15:39:02.568722+0000","last_clean":"2024-03-18T15:39:02.568722+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:39:02.568722+0000","last_undegraded":"2024-03-18T15:39:02.568722+0000","last_fullsized":"2024-03-18T15:39:02.568722+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610855+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:39:02.610855+0000","last_peered":"2024-03-18T15:39:02.610855+0000","last_clean":"2024-03-18T15:39:02.610855+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:39:02.610855+0000","last_undegraded":"2024-03-18T15:39:02.610855+0000","last_fullsized":"2024-03-18T15:39:02.610855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:21.223780+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:39:21.223780+0000","last_peered":"2024-03-18T15:39:21.223780+0000","last_clean":"2024-03-18T15:39:21.223780+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:39:21.223780+0000","last_undegraded":"2024-03-18T15:39:21.223780+0000","last_fullsized":"2024-03-18T15:39:21.223780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568391+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:39:02.568391+0000","last_peered":"2024-03-18T15:39:02.568391+0000","last_clean":"2024-03-18T15:39:02.568391+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:39:02.568391+0000","last_undegraded":"2024-03-18T15:39:02.568391+0000","last_fullsized":"2024-03-18T15:39:02.568391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609466+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:39:02.609466+0000","last_peered":"2024-03-18T15:39:02.609466+0000","last_clean":"2024-03-18T15:39:02.609466+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:39:02.609466+0000","last_undegraded":"2024-03-18T15:39:02.609466+0000","last_fullsized":"2024-03-18T15:39:02.609466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568601+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:39:02.568601+0000","last_peered":"2024-03-18T15:39:02.568601+0000","last_clean":"2024-03-18T15:39:02.568601+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:39:02.568601+0000","last_undegraded":"2024-03-18T15:39:02.568601+0000","last_fullsized":"2024-03-18T15:39:02.568601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609673+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:39:02.609673+0000","last_peered":"2024-03-18T15:39:02.609673+0000","last_clean":"2024-03-18T15:39:02.609673+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:39:02.609673+0000","last_undegraded":"2024-03-18T15:39:02.609673+0000","last_fullsized":"2024-03-18T15:39:02.609673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569145+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:39:02.569145+0000","last_peered":"2024-03-18T15:39:02.569145+0000","last_clean":"2024-03-18T15:39:02.569145+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:39:02.569145+0000","last_undegraded":"2024-03-18T15:39:02.569145+0000","last_fullsized":"2024-03-18T15:39:02.569145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568251+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:39:02.568251+0000","last_peered":"2024-03-18T15:39:02.568251+0000","last_clean":"2024-03-18T15:39:02.568251+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:39:02.568251+0000","last_undegraded":"2024-03-18T15:39:02.568251+0000","last_fullsized":"2024-03-18T15:39:02.568251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565656+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:39:02.565656+0000","last_peered":"2024-03-18T15:39:02.565656+0000","last_clean":"2024-03-18T15:39:02.565656+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:39:02.565656+0000","last_undegraded":"2024-03-18T15:39:02.565656+0000","last_fullsized":"2024-03-18T15:39:02.565656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":163,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:20.222395+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:39:20.222395+0000","last_peered":"2024-03-18T15:39:20.222395+0000","last_clean":"2024-03-18T15:39:20.222395+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:39:20.222395+0000","last_undegraded":"2024-03-18T15:39:20.222395+0000","last_fullsized":"2024-03-18T15:39:20.222395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609699+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:39:02.609699+0000","last_peered":"2024-03-18T15:39:02.609699+0000","last_clean":"2024-03-18T15:39:02.609699+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:39:02.609699+0000","last_undegraded":"2024-03-18T15:39:02.609699+0000","last_fullsized":"2024-03-18T15:39:02.609699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568499+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:39:02.568499+0000","last_peered":"2024-03-18T15:39:02.568499+0000","last_clean":"2024-03-18T15:39:02.568499+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:39:02.568499+0000","last_undegraded":"2024-03-18T15:39:02.568499+0000","last_fullsized":"2024-03-18T15:39:02.568499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.611014+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:39:02.611014+0000","last_peered":"2024-03-18T15:39:02.611014+0000","last_clean":"2024-03-18T15:39:02.611014+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:39:02.611014+0000","last_undegraded":"2024-03-18T15:39:02.611014+0000","last_fullsized":"2024-03-18T15:39:02.611014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569009+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:39:02.569009+0000","last_peered":"2024-03-18T15:39:02.569009+0000","last_clean":"2024-03-18T15:39:02.569009+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:39:02.569009+0000","last_undegraded":"2024-03-18T15:39:02.569009+0000","last_fullsized":"2024-03-18T15:39:02.569009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609527+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:39:02.609527+0000","last_peered":"2024-03-18T15:39:02.609527+0000","last_clean":"2024-03-18T15:39:02.609527+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:39:02.609527+0000","last_undegraded":"2024-03-18T15:39:02.609527+0000","last_fullsized":"2024-03-18T15:39:02.609527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":195,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:20.222380+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:39:20.222380+0000","last_peered":"2024-03-18T15:39:20.222380+0000","last_clean":"2024-03-18T15:39:20.222380+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:39:20.222380+0000","last_undegraded":"2024-03-18T15:39:20.222380+0000","last_fullsized":"2024-03-18T15:39:20.222380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":146,"num_read_kb":133,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568902+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:39:02.568902+0000","last_peered":"2024-03-18T15:39:02.568902+0000","last_clean":"2024-03-18T15:39:02.568902+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:39:02.568902+0000","last_undegraded":"2024-03-18T15:39:02.568902+0000","last_fullsized":"2024-03-18T15:39:02.568902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571905+0000","last_change":"2024-03-18T15:38:03.834819+0000","last_active":"2024-03-18T15:39:02.571905+0000","last_peered":"2024-03-18T15:39:02.571905+0000","last_clean":"2024-03-18T15:39:02.571905+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:39:02.571905+0000","last_undegraded":"2024-03-18T15:39:02.571905+0000","last_fullsized":"2024-03-18T15:39:02.571905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:02:11.448328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609821+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:39:02.609821+0000","last_peered":"2024-03-18T15:39:02.609821+0000","last_clean":"2024-03-18T15:39:02.609821+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:39:02.609821+0000","last_undegraded":"2024-03-18T15:39:02.609821+0000","last_fullsized":"2024-03-18T15:39:02.609821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568342+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:39:02.568342+0000","last_peered":"2024-03-18T15:39:02.568342+0000","last_clean":"2024-03-18T15:39:02.568342+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:39:02.568342+0000","last_undegraded":"2024-03-18T15:39:02.568342+0000","last_fullsized":"2024-03-18T15:39:02.568342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568642+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:39:02.568642+0000","last_peered":"2024-03-18T15:39:02.568642+0000","last_clean":"2024-03-18T15:39:02.568642+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:39:02.568642+0000","last_undegraded":"2024-03-18T15:39:02.568642+0000","last_fullsized":"2024-03-18T15:39:02.568642+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:22.226239+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:39:22.226239+0000","last_peered":"2024-03-18T15:39:22.226239+0000","last_clean":"2024-03-18T15:39:22.226239+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:39:22.226239+0000","last_undegraded":"2024-03-18T15:39:22.226239+0000","last_fullsized":"2024-03-18T15:39:22.226239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567566+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:39:02.567566+0000","last_peered":"2024-03-18T15:39:02.567566+0000","last_clean":"2024-03-18T15:39:02.567566+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:39:02.567566+0000","last_undegraded":"2024-03-18T15:39:02.567566+0000","last_fullsized":"2024-03-18T15:39:02.567566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567940+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:39:02.567940+0000","last_peered":"2024-03-18T15:39:02.567940+0000","last_clean":"2024-03-18T15:39:02.567940+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:39:02.567940+0000","last_undegraded":"2024-03-18T15:39:02.567940+0000","last_fullsized":"2024-03-18T15:39:02.567940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567932+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:39:02.567932+0000","last_peered":"2024-03-18T15:39:02.567932+0000","last_clean":"2024-03-18T15:39:02.567932+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:39:02.567932+0000","last_undegraded":"2024-03-18T15:39:02.567932+0000","last_fullsized":"2024-03-18T15:39:02.567932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.578097+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:39:02.578097+0000","last_peered":"2024-03-18T15:39:02.578097+0000","last_clean":"2024-03-18T15:39:02.578097+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:39:02.578097+0000","last_undegraded":"2024-03-18T15:39:02.578097+0000","last_fullsized":"2024-03-18T15:39:02.578097+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571921+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:39:02.571921+0000","last_peered":"2024-03-18T15:39:02.571921+0000","last_clean":"2024-03-18T15:39:02.571921+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:39:02.571921+0000","last_undegraded":"2024-03-18T15:39:02.571921+0000","last_fullsized":"2024-03-18T15:39:02.571921+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565718+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:39:02.565718+0000","last_peered":"2024-03-18T15:39:02.565718+0000","last_clean":"2024-03-18T15:39:02.565718+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:39:02.565718+0000","last_undegraded":"2024-03-18T15:39:02.565718+0000","last_fullsized":"2024-03-18T15:39:02.565718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609902+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:39:02.609902+0000","last_peered":"2024-03-18T15:39:02.609902+0000","last_clean":"2024-03-18T15:39:02.609902+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:39:02.609902+0000","last_undegraded":"2024-03-18T15:39:02.609902+0000","last_fullsized":"2024-03-18T15:39:02.609902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:17.221139+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:39:17.221139+0000","last_peered":"2024-03-18T15:39:17.221139+0000","last_clean":"2024-03-18T15:39:17.221139+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:39:17.221139+0000","last_undegraded":"2024-03-18T15:39:17.221139+0000","last_fullsized":"2024-03-18T15:39:17.221139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609806+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:39:02.609806+0000","last_peered":"2024-03-18T15:39:02.609806+0000","last_clean":"2024-03-18T15:39:02.609806+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:39:02.609806+0000","last_undegraded":"2024-03-18T15:39:02.609806+0000","last_fullsized":"2024-03-18T15:39:02.609806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568433+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:39:02.568433+0000","last_peered":"2024-03-18T15:39:02.568433+0000","last_clean":"2024-03-18T15:39:02.568433+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:39:02.568433+0000","last_undegraded":"2024-03-18T15:39:02.568433+0000","last_fullsized":"2024-03-18T15:39:02.568433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:21.224820+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:39:21.224820+0000","last_peered":"2024-03-18T15:39:21.224820+0000","last_clean":"2024-03-18T15:39:21.224820+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:39:21.224820+0000","last_undegraded":"2024-03-18T15:39:21.224820+0000","last_fullsized":"2024-03-18T15:39:21.224820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610861+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:39:02.610861+0000","last_peered":"2024-03-18T15:39:02.610861+0000","last_clean":"2024-03-18T15:39:02.610861+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:39:02.610861+0000","last_undegraded":"2024-03-18T15:39:02.610861+0000","last_fullsized":"2024-03-18T15:39:02.610861+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567818+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:39:02.567818+0000","last_peered":"2024-03-18T15:39:02.567818+0000","last_clean":"2024-03-18T15:39:02.567818+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:39:02.567818+0000","last_undegraded":"2024-03-18T15:39:02.567818+0000","last_fullsized":"2024-03-18T15:39:02.567818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567855+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:39:02.567855+0000","last_peered":"2024-03-18T15:39:02.567855+0000","last_clean":"2024-03-18T15:39:02.567855+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:39:02.567855+0000","last_undegraded":"2024-03-18T15:39:02.567855+0000","last_fullsized":"2024-03-18T15:39:02.567855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609547+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:39:02.609547+0000","last_peered":"2024-03-18T15:39:02.609547+0000","last_clean":"2024-03-18T15:39:02.609547+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:39:02.609547+0000","last_undegraded":"2024-03-18T15:39:02.609547+0000","last_fullsized":"2024-03-18T15:39:02.609547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.567978+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:39:02.567978+0000","last_peered":"2024-03-18T15:39:02.567978+0000","last_clean":"2024-03-18T15:39:02.567978+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:39:02.567978+0000","last_undegraded":"2024-03-18T15:39:02.567978+0000","last_fullsized":"2024-03-18T15:39:02.567978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568992+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:39:02.568992+0000","last_peered":"2024-03-18T15:39:02.568992+0000","last_clean":"2024-03-18T15:39:02.568992+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:39:02.568992+0000","last_undegraded":"2024-03-18T15:39:02.568992+0000","last_fullsized":"2024-03-18T15:39:02.568992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609980+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:39:02.609980+0000","last_peered":"2024-03-18T15:39:02.609980+0000","last_clean":"2024-03-18T15:39:02.609980+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:39:02.609980+0000","last_undegraded":"2024-03-18T15:39:02.609980+0000","last_fullsized":"2024-03-18T15:39:02.609980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.566128+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:39:02.566128+0000","last_peered":"2024-03-18T15:39:02.566128+0000","last_clean":"2024-03-18T15:39:02.566128+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:39:02.566128+0000","last_undegraded":"2024-03-18T15:39:02.566128+0000","last_fullsized":"2024-03-18T15:39:02.566128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569239+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:39:02.569239+0000","last_peered":"2024-03-18T15:39:02.569239+0000","last_clean":"2024-03-18T15:39:02.569239+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:39:02.569239+0000","last_undegraded":"2024-03-18T15:39:02.569239+0000","last_fullsized":"2024-03-18T15:39:02.569239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568297+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:39:02.568297+0000","last_peered":"2024-03-18T15:39:02.568297+0000","last_clean":"2024-03-18T15:39:02.568297+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:39:02.568297+0000","last_undegraded":"2024-03-18T15:39:02.568297+0000","last_fullsized":"2024-03-18T15:39:02.568297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569289+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:39:02.569289+0000","last_peered":"2024-03-18T15:39:02.569289+0000","last_clean":"2024-03-18T15:39:02.569289+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:39:02.569289+0000","last_undegraded":"2024-03-18T15:39:02.569289+0000","last_fullsized":"2024-03-18T15:39:02.569289+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609419+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:39:02.609419+0000","last_peered":"2024-03-18T15:39:02.609419+0000","last_clean":"2024-03-18T15:39:02.609419+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:39:02.609419+0000","last_undegraded":"2024-03-18T15:39:02.609419+0000","last_fullsized":"2024-03-18T15:39:02.609419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568062+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:39:02.568062+0000","last_peered":"2024-03-18T15:39:02.568062+0000","last_clean":"2024-03-18T15:39:02.568062+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:39:02.568062+0000","last_undegraded":"2024-03-18T15:39:02.568062+0000","last_fullsized":"2024-03-18T15:39:02.568062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569256+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:39:02.569256+0000","last_peered":"2024-03-18T15:39:02.569256+0000","last_clean":"2024-03-18T15:39:02.569256+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:39:02.569256+0000","last_undegraded":"2024-03-18T15:39:02.569256+0000","last_fullsized":"2024-03-18T15:39:02.569256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565066+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:39:02.565066+0000","last_peered":"2024-03-18T15:39:02.565066+0000","last_clean":"2024-03-18T15:39:02.565066+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:39:02.565066+0000","last_undegraded":"2024-03-18T15:39:02.565066+0000","last_fullsized":"2024-03-18T15:39:02.565066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570820+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:39:02.570820+0000","last_peered":"2024-03-18T15:39:02.570820+0000","last_clean":"2024-03-18T15:39:02.570820+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:39:02.570820+0000","last_undegraded":"2024-03-18T15:39:02.570820+0000","last_fullsized":"2024-03-18T15:39:02.570820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569730+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:39:02.569730+0000","last_peered":"2024-03-18T15:39:02.569730+0000","last_clean":"2024-03-18T15:39:02.569730+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:39:02.569730+0000","last_undegraded":"2024-03-18T15:39:02.569730+0000","last_fullsized":"2024-03-18T15:39:02.569730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.581338+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:39:02.581338+0000","last_peered":"2024-03-18T15:39:02.581338+0000","last_clean":"2024-03-18T15:39:02.581338+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:39:02.581338+0000","last_undegraded":"2024-03-18T15:39:02.581338+0000","last_fullsized":"2024-03-18T15:39:02.581338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568086+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:39:02.568086+0000","last_peered":"2024-03-18T15:39:02.568086+0000","last_clean":"2024-03-18T15:39:02.568086+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:39:02.568086+0000","last_undegraded":"2024-03-18T15:39:02.568086+0000","last_fullsized":"2024-03-18T15:39:02.568086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:22.238483+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:39:22.238483+0000","last_peered":"2024-03-18T15:39:22.238483+0000","last_clean":"2024-03-18T15:39:22.238483+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:39:22.238483+0000","last_undegraded":"2024-03-18T15:39:22.238483+0000","last_fullsized":"2024-03-18T15:39:22.238483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568443+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:39:02.568443+0000","last_peered":"2024-03-18T15:39:02.568443+0000","last_clean":"2024-03-18T15:39:02.568443+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:39:02.568443+0000","last_undegraded":"2024-03-18T15:39:02.568443+0000","last_fullsized":"2024-03-18T15:39:02.568443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568502+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:39:02.568502+0000","last_peered":"2024-03-18T15:39:02.568502+0000","last_clean":"2024-03-18T15:39:02.568502+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:39:02.568502+0000","last_undegraded":"2024-03-18T15:39:02.568502+0000","last_fullsized":"2024-03-18T15:39:02.568502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568235+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:39:02.568235+0000","last_peered":"2024-03-18T15:39:02.568235+0000","last_clean":"2024-03-18T15:39:02.568235+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:39:02.568235+0000","last_undegraded":"2024-03-18T15:39:02.568235+0000","last_fullsized":"2024-03-18T15:39:02.568235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609878+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:39:02.609878+0000","last_peered":"2024-03-18T15:39:02.609878+0000","last_clean":"2024-03-18T15:39:02.609878+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:39:02.609878+0000","last_undegraded":"2024-03-18T15:39:02.609878+0000","last_fullsized":"2024-03-18T15:39:02.609878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568293+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:39:02.568293+0000","last_peered":"2024-03-18T15:39:02.568293+0000","last_clean":"2024-03-18T15:39:02.568293+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:39:02.568293+0000","last_undegraded":"2024-03-18T15:39:02.568293+0000","last_fullsized":"2024-03-18T15:39:02.568293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565134+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:39:02.565134+0000","last_peered":"2024-03-18T15:39:02.565134+0000","last_clean":"2024-03-18T15:39:02.565134+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:39:02.565134+0000","last_undegraded":"2024-03-18T15:39:02.565134+0000","last_fullsized":"2024-03-18T15:39:02.565134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.571006+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:39:02.571006+0000","last_peered":"2024-03-18T15:39:02.571006+0000","last_clean":"2024-03-18T15:39:02.571006+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:39:02.571006+0000","last_undegraded":"2024-03-18T15:39:02.571006+0000","last_fullsized":"2024-03-18T15:39:02.571006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.574194+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:39:02.574194+0000","last_peered":"2024-03-18T15:39:02.574194+0000","last_clean":"2024-03-18T15:39:02.574194+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:39:02.574194+0000","last_undegraded":"2024-03-18T15:39:02.574194+0000","last_fullsized":"2024-03-18T15:39:02.574194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569500+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:39:02.569500+0000","last_peered":"2024-03-18T15:39:02.569500+0000","last_clean":"2024-03-18T15:39:02.569500+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:39:02.569500+0000","last_undegraded":"2024-03-18T15:39:02.569500+0000","last_fullsized":"2024-03-18T15:39:02.569500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609768+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:39:02.609768+0000","last_peered":"2024-03-18T15:39:02.609768+0000","last_clean":"2024-03-18T15:39:02.609768+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:39:02.609768+0000","last_undegraded":"2024-03-18T15:39:02.609768+0000","last_fullsized":"2024-03-18T15:39:02.609768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.565343+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:39:02.565343+0000","last_peered":"2024-03-18T15:39:02.565343+0000","last_clean":"2024-03-18T15:39:02.565343+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:39:02.565343+0000","last_undegraded":"2024-03-18T15:39:02.565343+0000","last_fullsized":"2024-03-18T15:39:02.565343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.570696+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:39:02.570696+0000","last_peered":"2024-03-18T15:39:02.570696+0000","last_clean":"2024-03-18T15:39:02.570696+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:39:02.570696+0000","last_undegraded":"2024-03-18T15:39:02.570696+0000","last_fullsized":"2024-03-18T15:39:02.570696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569117+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:39:02.569117+0000","last_peered":"2024-03-18T15:39:02.569117+0000","last_clean":"2024-03-18T15:39:02.569117+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:39:02.569117+0000","last_undegraded":"2024-03-18T15:39:02.569117+0000","last_fullsized":"2024-03-18T15:39:02.569117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.568348+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:39:02.568348+0000","last_peered":"2024-03-18T15:39:02.568348+0000","last_clean":"2024-03-18T15:39:02.568348+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:39:02.568348+0000","last_undegraded":"2024-03-18T15:39:02.568348+0000","last_fullsized":"2024-03-18T15:39:02.568348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569077+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:39:02.569077+0000","last_peered":"2024-03-18T15:39:02.569077+0000","last_clean":"2024-03-18T15:39:02.569077+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:39:02.569077+0000","last_undegraded":"2024-03-18T15:39:02.569077+0000","last_fullsized":"2024-03-18T15:39:02.569077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.609471+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:39:02.609471+0000","last_peered":"2024-03-18T15:39:02.609471+0000","last_clean":"2024-03-18T15:39:02.609471+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:39:02.609471+0000","last_undegraded":"2024-03-18T15:39:02.609471+0000","last_fullsized":"2024-03-18T15:39:02.609471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.610015+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:39:02.610015+0000","last_peered":"2024-03-18T15:39:02.610015+0000","last_clean":"2024-03-18T15:39:02.610015+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:39:02.610015+0000","last_undegraded":"2024-03-18T15:39:02.610015+0000","last_fullsized":"2024-03-18T15:39:02.610015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:02.569338+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:39:02.569338+0000","last_peered":"2024-03-18T15:39:02.569338+0000","last_clean":"2024-03-18T15:39:02.569338+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:39:02.569338+0000","last_undegraded":"2024-03-18T15:39:02.569338+0000","last_fullsized":"2024-03-18T15:39:02.569338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67468,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":742,"num_read_kb":443,"num_write":235,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":202404,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":461450,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":1113503,"data_compressed":43320,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":461358,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037805,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":461714,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1053863,"data_compressed":41452,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":461075,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":523034,"data_compressed":7723,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":462414,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":523254,"data_compressed":7723,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":460900,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460936,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724605,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1053541,"data_compressed":41452,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62782,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:39:25.752 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-18T15:39:25.752 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T15:39:25.752 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-18T15:39:25.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-18T15:39:25.995 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:26.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:26 smithi096 ceph-mon[31930]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:26.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:26 smithi096 ceph-mon[31930]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:26.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:26 smithi137 ceph-mon[35868]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:26.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:26 smithi137 ceph-mon[35868]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:26.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:26 smithi193 ceph-mon[36702]: from='client.24832 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:39:26.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:26 smithi193 ceph-mon[36702]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 -- 172.21.15.96:0/2313794854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 msgr2=0x7f21f410e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 --2- 172.21.15.96:0/2313794854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f410e100 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b0d0 tx=0x7f21ec00b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 -- 172.21.15.96:0/2313794854 shutdown_connections 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 --2- 172.21.15.96:0/2313794854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f410e100 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 --2- 172.21.15.96:0/2313794854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101450 0x7f21f4101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 --2- 172.21.15.96:0/2313794854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101e00 0x7f21f4105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 -- 172.21.15.96:0/2313794854 >> 172.21.15.96:0/2313794854 conn(0x7f21f4078230 msgr2=0x7f21f4078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 -- 172.21.15.96:0/2313794854 shutdown_connections 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.668+0000 7f21fc728700 1 -- 172.21.15.96:0/2313794854 wait complete. 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 Processor -- start 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 -- start start 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 0x7f21f41a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:27.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 0x7f21f41a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111540 con 0x7f21f4106530 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41113c0 con 0x7f21f4101450 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fc728700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41116c0 con 0x7f21f4101e00 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21f9cc3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 0x7f21f41a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21f9cc3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 0x7f21f41a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39024/0 (socket says 172.21.15.96:39024) 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36538/0 (socket says 172.21.15.96:36538) 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21f9cc3700 1 -- 172.21.15.96:0/3381648433 learned_addr learned my addr 172.21.15.96:0/3381648433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fa4c4700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 0x7f21f41a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 -- 172.21.15.96:0/3381648433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 msgr2=0x7f21f41a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 0x7f21f41a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 -- 172.21.15.96:0/3381648433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 msgr2=0x7f21f41a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 0x7f21f41a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21facc5700 1 -- 172.21.15.96:0/3381648433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f419b1f0 con 0x7f21f4106530 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.669+0000 7f21fa4c4700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 0x7f21f41a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:27.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21facc5700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b770 tx=0x7f21ec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec021020 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec004c50 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f419b480 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0052b0 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.670+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a77a0 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.671+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4102930 con 0x7f21f4106530 2024-03-18T15:39:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.672+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f21ec024020 con 0x7f21f4106530 2024-03-18T15:39:27.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.673+0000 7f21eb7fe700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 0x7f21e00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:27.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.673+0000 7f21fa4c4700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 0x7f21e00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:27.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.673+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f21ec0cf8f0 con 0x7f21f4106530 2024-03-18T15:39:27.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.674+0000 7f21fa4c4700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 0x7f21e00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21f419bbd0 tx=0x7f21f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:27.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.678+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f21ec02b030 con 0x7f21f4106530 2024-03-18T15:39:27.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.921+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f21f419bfc0 con 0x7f21f4106530 2024-03-18T15:39:27.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.921+0000 7f21eb7fe700 1 -- 172.21.15.96:0/3381648433 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f21ec01de20 con 0x7f21f4106530 2024-03-18T15:39:27.921 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:27.921 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T15:39:27.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 msgr2=0x7f21e00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:27.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 0x7f21e00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21f419bbd0 tx=0x7f21f0009260 comp rx=0 tx=0).stop 2024-03-18T15:39:27.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 msgr2=0x7f21f419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:27.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b770 tx=0x7f21ec007a20 comp rx=0 tx=0).stop 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 shutdown_connections 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f21e00a2920 0x7f21e00a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21f4106530 0x7f21f419aa90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f21f4101450 0x7f21f41a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 --2- 172.21.15.96:0/3381648433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21f4101e00 0x7f21f41a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.923+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 >> 172.21.15.96:0/3381648433 conn(0x7f21f4078230 msgr2=0x7f21f410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.924+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 shutdown_connections 2024-03-18T15:39:27.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:27.924+0000 7f21fc728700 1 -- 172.21.15.96:0/3381648433 wait complete. 2024-03-18T15:39:28.592 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-18T15:39:28.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:28.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 -- 172.21.15.96:0/1336252308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82741029f0 msgr2=0x7f8274102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:28.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 --2- 172.21.15.96:0/1336252308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82741029f0 0x7f8274102e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f825c009900 tx=0x7f825c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:28.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 -- 172.21.15.96:0/1336252308 shutdown_connections 2024-03-18T15:39:28.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 --2- 172.21.15.96:0/1336252308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741033b0 0x7f8274109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 --2- 172.21.15.96:0/1336252308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82741029f0 0x7f8274102e70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 --2- 172.21.15.96:0/1336252308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8274101800 0x7f8274101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 -- 172.21.15.96:0/1336252308 >> 172.21.15.96:0/1336252308 conn(0x7f82740fcd90 msgr2=0x7f82740ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 -- 172.21.15.96:0/1336252308 shutdown_connections 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.697+0000 7f8278f5b640 1 -- 172.21.15.96:0/1336252308 wait complete. 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 Processor -- start 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 -- start start 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741029f0 0x7f8274198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82741033b0 0x7f827419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f827410d460 con 0x7f8274101800 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f827410d2e0 con 0x7f82741033b0 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8278f5b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f827410d5e0 con 0x7f82741029f0 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36558/0 (socket says 172.21.15.96:36558) 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 -- 172.21.15.96:0/877329174 learned_addr learned my addr 172.21.15.96:0/877329174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f826bfff640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741029f0 0x7f8274198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f8272d9d640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82741033b0 0x7f827419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 -- 172.21.15.96:0/877329174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741029f0 msgr2=0x7f8274198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741029f0 0x7f8274198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 -- 172.21.15.96:0/877329174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82741033b0 msgr2=0x7f827419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82741033b0 0x7f827419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 -- 172.21.15.96:0/877329174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827419f720 con 0x7f8274101800 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.698+0000 7f827259c640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9c0 tx=0x7f826400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.699+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826400cd80 con 0x7f8274101800 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.699+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82640035f0 con 0x7f8274101800 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.699+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827419fa10 con 0x7f8274101800 2024-03-18T15:39:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.699+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264010690 con 0x7f8274101800 2024-03-18T15:39:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.699+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827419ff30 con 0x7f8274101800 2024-03-18T15:39:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.700+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274108b20 con 0x7f8274101800 2024-03-18T15:39:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.701+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f82640037a0 con 0x7f8274101800 2024-03-18T15:39:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.702+0000 7f826b7fe640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 0x7f82600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.702+0000 7f826bfff640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 0x7f82600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.702+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f82640ceff0 con 0x7f8274101800 2024-03-18T15:39:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.702+0000 7f826bfff640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 0x7f82600a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f825c02c7b0 tx=0x7f825c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.705+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f826409c050 con 0x7f8274101800 2024-03-18T15:39:28.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:28 smithi137 ceph-mon[35868]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:28.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3381648433' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T15:39:28.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.881+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f827402d1b0 con 0x7f8274101800 2024-03-18T15:39:28.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.883+0000 7f826b7fe640 1 -- 172.21.15.96:0/877329174 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7f826409b950 con 0x7f8274101800 2024-03-18T15:39:28.884 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:28.884 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 msgr2=0x7f82600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 0x7f82600a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f825c02c7b0 tx=0x7f825c005fd0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 msgr2=0x7f8274197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f826400e9c0 tx=0x7f826400ee80 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 shutdown_connections 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82600a2890 0x7f82600a4d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82741033b0 0x7f827419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82741029f0 0x7f8274198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 --2- 172.21.15.96:0/877329174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8274101800 0x7f8274197b00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 >> 172.21.15.96:0/877329174 conn(0x7f82740fcd90 msgr2=0x7f8274105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:28.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 shutdown_connections 2024-03-18T15:39:28.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:28.885+0000 7f8278f5b640 1 -- 172.21.15.96:0/877329174 wait complete. 2024-03-18T15:39:28.896 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:28 smithi096 ceph-mon[31930]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:28.896 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3381648433' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T15:39:28.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:39:28.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:39:28.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:28 smithi193 ceph-mon[36702]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3381648433' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 -- 172.21.15.96:0/360941424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022a0 msgr2=0x7f6508108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 --2- 172.21.15.96:0/360941424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022a0 0x7f6508108b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6500009ca0 tx=0x7f650002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 -- 172.21.15.96:0/360941424 shutdown_connections 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 --2- 172.21.15.96:0/360941424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022a0 0x7f6508108b30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 --2- 172.21.15.96:0/360941424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081018e0 0x7f6508101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 --2- 172.21.15.96:0/360941424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081006f0 0x7f6508100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 -- 172.21.15.96:0/360941424 >> 172.21.15.96:0/360941424 conn(0x7f65080fbc80 msgr2=0x7f65080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 -- 172.21.15.96:0/360941424 shutdown_connections 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 -- 172.21.15.96:0/360941424 wait complete. 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.001+0000 7f650d9d4640 1 Processor -- start 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- start start 2024-03-18T15:39:29.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081006f0 0x7f6508197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081022a0 0x7f650819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d450 con 0x7f65081018e0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d2d0 con 0x7f65081022a0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f650810d5d0 con 0x7f65081006f0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f6506ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081006f0 0x7f6508197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36586/0 (socket says 172.21.15.96:36586) 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 -- 172.21.15.96:0/4022995184 learned_addr learned my addr 172.21.15.96:0/4022995184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f65077fe640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081022a0 0x7f650819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 -- 172.21.15.96:0/4022995184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081006f0 msgr2=0x7f6508197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081006f0 0x7f6508197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 -- 172.21.15.96:0/4022995184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081022a0 msgr2=0x7f650819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081022a0 0x7f650819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 -- 172.21.15.96:0/4022995184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650819f6e0 con 0x7f65081018e0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f64fffff640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ea60 tx=0x7f64f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f000cde0 con 0x7f65081018e0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f00035f0 con 0x7f65081018e0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650819f9d0 con 0x7f65081018e0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0010670 con 0x7f65081018e0 2024-03-18T15:39:29.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.002+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650819ff20 con 0x7f65081018e0 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.003+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6508107a10 con 0x7f65081018e0 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.004+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f64f00037a0 con 0x7f65081018e0 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.005+0000 7f650c9d2640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 0x7f64f40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.005+0000 7f6506ffd640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 0x7f64f40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.005+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f64f00ceef0 con 0x7f65081018e0 2024-03-18T15:39:29.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.005+0000 7f6506ffd640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 0x7f64f40a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6508101740 tx=0x7f64f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.009+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f64f009be20 con 0x7f65081018e0 2024-03-18T15:39:29.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.184+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f650802d3b0 con 0x7f65081018e0 2024-03-18T15:39:29.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.184+0000 7f650c9d2640 1 -- 172.21.15.96:0/4022995184 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f64f009b720 con 0x7f65081018e0 2024-03-18T15:39:29.185 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:39:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 msgr2=0x7f64f40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 0x7f64f40a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6508101740 tx=0x7f64f8005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 msgr2=0x7f6508198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f64f000ea60 tx=0x7f64f000ef20 comp rx=0 tx=0).stop 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 shutdown_connections 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f64f40a2890 0x7f64f40a4d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65081022a0 0x7f650819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081018e0 0x7f6508198000 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 --2- 172.21.15.96:0/4022995184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65081006f0 0x7f6508197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 >> 172.21.15.96:0/4022995184 conn(0x7f65080fbc80 msgr2=0x7f6508104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 shutdown_connections 2024-03-18T15:39:29.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.186+0000 7f650d9d4640 1 -- 172.21.15.96:0/4022995184 wait complete. 2024-03-18T15:39:29.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- 172.21.15.96:0/2422706603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601006f0 msgr2=0x7f8a60100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- 172.21.15.96:0/2422706603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601006f0 0x7f8a60100b10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009900 tx=0x7f8a5002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- 172.21.15.96:0/2422706603 shutdown_connections 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- 172.21.15.96:0/2422706603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 0x7f8a60108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- 172.21.15.96:0/2422706603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601018e0 0x7f8a60101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- 172.21.15.96:0/2422706603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601006f0 0x7f8a60100b10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- 172.21.15.96:0/2422706603 >> 172.21.15.96:0/2422706603 conn(0x7f8a600fbc80 msgr2=0x7f8a600fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- 172.21.15.96:0/2422706603 shutdown_connections 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- 172.21.15.96:0/2422706603 wait complete. 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 Processor -- start 2024-03-18T15:39:29.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- start start 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601006f0 0x7f8a60197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 0x7f8a6019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6010d490 con 0x7f8a601018e0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6010d310 con 0x7f8a601022a0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.302+0000 7f8a6833b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6010d610 con 0x7f8a601006f0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36600/0 (socket says 172.21.15.96:36600) 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 -- 172.21.15.96:0/1643630187 learned_addr learned my addr 172.21.15.96:0/1643630187 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a660d7640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601006f0 0x7f8a60197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a668d8640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 0x7f8a6019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 -- 172.21.15.96:0/1643630187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601006f0 msgr2=0x7f8a60197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601006f0 0x7f8a60197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 -- 172.21.15.96:0/1643630187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 msgr2=0x7f8a6019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 0x7f8a6019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 -- 172.21.15.96:0/1643630187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6019f770 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a658d6640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e9f0 tx=0x7f8a5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00cd80 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5c005500 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c010690 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6019fa60 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.303+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6019ff60 con 0x7f8a601018e0 2024-03-18T15:39:29.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.304+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a60107a90 con 0x7f8a601018e0 2024-03-18T15:39:29.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.305+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f8a5c0046c0 con 0x7f8a601018e0 2024-03-18T15:39:29.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.306+0000 7f8a577fe640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 0x7f8a4c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.306+0000 7f8a660d7640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 0x7f8a4c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.306+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f8a5c0cf1d0 con 0x7f8a601018e0 2024-03-18T15:39:29.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.306+0000 7f8a660d7640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 0x7f8a4c0a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.309+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8a5c053740 con 0x7f8a601018e0 2024-03-18T15:39:29.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.484+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8a6002d340 con 0x7f8a601018e0 2024-03-18T15:39:29.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.485+0000 7f8a577fe640 1 -- 172.21.15.96:0/1643630187 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f8a5c09c100 con 0x7f8a601018e0 2024-03-18T15:39:29.486 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 msgr2=0x7f8a4c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 0x7f8a4c0a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a50005fd0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 msgr2=0x7f8a60198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e9f0 tx=0x7f8a5c00edf0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 shutdown_connections 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a4c0a2890 0x7f8a4c0a4d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a601022a0 0x7f8a6019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a601018e0 0x7f8a60198090 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 --2- 172.21.15.96:0/1643630187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a601006f0 0x7f8a60197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 >> 172.21.15.96:0/1643630187 conn(0x7f8a600fbc80 msgr2=0x7f8a60104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 shutdown_connections 2024-03-18T15:39:29.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.487+0000 7f8a6833b640 1 -- 172.21.15.96:0/1643630187 wait complete. 2024-03-18T15:39:29.499 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- 172.21.15.96:0/4253715541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 msgr2=0x7f77c0109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/4253715541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c0109930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009ca0 tx=0x7f77b802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- 172.21.15.96:0/4253715541 shutdown_connections 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/4253715541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c0109930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/4253715541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c0104c10 0x7f77c0107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/4253715541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c01022e0 0x7f77c01046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- 172.21.15.96:0/4253715541 >> 172.21.15.96:0/4253715541 conn(0x7f77c00fbc90 msgr2=0x7f77c00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- 172.21.15.96:0/4253715541 shutdown_connections 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- 172.21.15.96:0/4253715541 wait complete. 2024-03-18T15:39:29.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 Processor -- start 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.604+0000 7f77c5dc4640 1 -- start start 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c01022e0 0x7f77c0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 0x7f77c01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d480 con 0x7f77c0107540 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d300 con 0x7f77c01022e0 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f77c010d600 con 0x7f77c0104c10 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36632/0 (socket says 172.21.15.96:36632) 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 -- 172.21.15.96:0/2410709291 learned_addr learned my addr 172.21.15.96:0/2410709291 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bf7fe640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c01022e0 0x7f77c0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77b6ffd640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 0x7f77c01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 -- 172.21.15.96:0/2410709291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 msgr2=0x7f77c01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 0x7f77c01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 -- 172.21.15.96:0/2410709291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c01022e0 msgr2=0x7f77c0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c01022e0 0x7f77c0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 -- 172.21.15.96:0/2410709291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c019f720 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77b6ffd640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 0x7f77c01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bffff640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f77b8037040 tx=0x7f77b8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8041020 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b8034cd0 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c019f9b0 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8039040 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.605+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c019fea0 con 0x7f77c0107540 2024-03-18T15:39:29.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.606+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c00fd780 con 0x7f77c0107540 2024-03-18T15:39:29.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.607+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f77b8044020 con 0x7f77c0107540 2024-03-18T15:39:29.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.608+0000 7f77bd7fa640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 0x7f77ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.608+0000 7f77bf7fe640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 0x7f77ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.608+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f77b80f0e00 con 0x7f77c0107540 2024-03-18T15:39:29.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.608+0000 7f77bf7fe640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 0x7f77ac0a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009bd0 tx=0x7f77b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.612+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f77b804b020 con 0x7f77c0107540 2024-03-18T15:39:29.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.787+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77c002d340 con 0x7f77c0107540 2024-03-18T15:39:29.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.787+0000 7f77bd7fa640 1 -- 172.21.15.96:0/2410709291 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f77b80bded0 con 0x7f77c0107540 2024-03-18T15:39:29.788 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 msgr2=0x7f77ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 0x7f77ac0a4d50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009bd0 tx=0x7f77b0005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 msgr2=0x7f77c019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f77b8037040 tx=0x7f77b8032f40 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 shutdown_connections 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f77ac0a2890 0x7f77ac0a4d50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77c0107540 0x7f77c019f0b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77c0104c10 0x7f77c01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 --2- 172.21.15.96:0/2410709291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f77c01022e0 0x7f77c0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 >> 172.21.15.96:0/2410709291 conn(0x7f77c00fbc90 msgr2=0x7f77c0102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 shutdown_connections 2024-03-18T15:39:29.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.789+0000 7f77c5dc4640 1 -- 172.21.15.96:0/2410709291 wait complete. 2024-03-18T15:39:29.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/877329174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:29.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4022995184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:29.800 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1643630187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:29.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:39:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/877329174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4022995184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:29.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1643630187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 -- 172.21.15.96:0/328321537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6338100fe0 msgr2=0x7f6338101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 --2- 172.21.15.96:0/328321537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6338100fe0 0x7f6338101460 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6328009900 tx=0x7f632802bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 -- 172.21.15.96:0/328321537 shutdown_connections 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 --2- 172.21.15.96:0/328321537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63381019a0 0x7f63381082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 --2- 172.21.15.96:0/328321537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6338100fe0 0x7f6338101460 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 --2- 172.21.15.96:0/328321537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63380ffe90 0x7f63381002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 -- 172.21.15.96:0/328321537 >> 172.21.15.96:0/328321537 conn(0x7f63380fb400 msgr2=0x7f63380fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 -- 172.21.15.96:0/328321537 shutdown_connections 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.901+0000 7f634013b640 1 -- 172.21.15.96:0/328321537 wait complete. 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 Processor -- start 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 -- start start 2024-03-18T15:39:29.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63380ffe90 0x7f6338197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338100fe0 0x7f6338198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f633810b2c0 con 0x7f63381019a0 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f633810b140 con 0x7f6338100fe0 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f634013b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f633810b440 con 0x7f63380ffe90 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36656/0 (socket says 172.21.15.96:36656) 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 -- 172.21.15.96:0/978183458 learned_addr learned my addr 172.21.15.96:0/978183458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633ded7640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63380ffe90 0x7f6338197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633d6d6640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338100fe0 0x7f6338198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 -- 172.21.15.96:0/978183458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63380ffe90 msgr2=0x7f6338197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63380ffe90 0x7f6338197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 -- 172.21.15.96:0/978183458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338100fe0 msgr2=0x7f6338198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338100fe0 0x7f6338198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:29.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 -- 172.21.15.96:0/978183458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633819f740 con 0x7f63381019a0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f633e6d8640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f633000ec60 tx=0x7f633000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.902+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330004d20 con 0x7f63381019a0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.903+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633819fa30 con 0x7f63381019a0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.903+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633000ba50 con 0x7f63381019a0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.903+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330005740 con 0x7f63381019a0 2024-03-18T15:39:29.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.903+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633819ffa0 con 0x7f63381019a0 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.903+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63381a02c0 con 0x7f63381019a0 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.904+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f63300058e0 con 0x7f63381019a0 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.905+0000 7f632effd640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 0x7f63240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.905+0000 7f633ded7640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 0x7f63240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.905+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f63300cf000 con 0x7f63381019a0 2024-03-18T15:39:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.905+0000 7f633ded7640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 0x7f63240a4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6334009bd0 tx=0x7f6334005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:29.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:29.910+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f633009bf30 con 0x7f63381019a0 2024-03-18T15:39:29.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/877329174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:29.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4022995184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:29.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1643630187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:30.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.121+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63381a0490 con 0x7f63381019a0 2024-03-18T15:39:30.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.122+0000 7f632effd640 1 -- 172.21.15.96:0/978183458 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f633009b830 con 0x7f63381019a0 2024-03-18T15:39:30.123 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:30.123 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:39:30.123 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 msgr2=0x7f63240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 0x7f63240a4d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6334009bd0 tx=0x7f6334005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 msgr2=0x7f633819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f633000ec60 tx=0x7f633000c7a0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 shutdown_connections 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63240a2890 0x7f63240a4d50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63381019a0 0x7f633819f0d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6338100fe0 0x7f6338198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 --2- 172.21.15.96:0/978183458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63380ffe90 0x7f6338197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 >> 172.21.15.96:0/978183458 conn(0x7f63380fb400 msgr2=0x7f6338106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 shutdown_connections 2024-03-18T15:39:30.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.124+0000 7f634013b640 1 -- 172.21.15.96:0/978183458 wait complete. 2024-03-18T15:39:30.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:30.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 -- 172.21.15.96:0/3745898752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b50102a00 msgr2=0x7f2b50102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 --2- 172.21.15.96:0/3745898752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b50102a00 0x7f2b50102e80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009900 tx=0x7f2b4402bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 -- 172.21.15.96:0/3745898752 shutdown_connections 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 --2- 172.21.15.96:0/3745898752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b501033c0 0x7f2b50109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 --2- 172.21.15.96:0/3745898752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b50102a00 0x7f2b50102e80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 --2- 172.21.15.96:0/3745898752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 0x7f2b50101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 -- 172.21.15.96:0/3745898752 >> 172.21.15.96:0/3745898752 conn(0x7f2b500fcd80 msgr2=0x7f2b500ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 -- 172.21.15.96:0/3745898752 shutdown_connections 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.237+0000 7f2b56627640 1 -- 172.21.15.96:0/3745898752 wait complete. 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 Processor -- start 2024-03-18T15:39:30.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 -- start start 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 0x7f2b50197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50102a00 0x7f2b50198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5010d490 con 0x7f2b501033c0 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5010d310 con 0x7f2b50102a00 2024-03-18T15:39:30.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b56627640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5010d610 con 0x7f2b50101810 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b54bc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b54bc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36664/0 (socket says 172.21.15.96:36664) 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b54bc4640 1 -- 172.21.15.96:0/318078634 learned_addr learned my addr 172.21.15.96:0/318078634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b4ffff640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 0x7f2b50197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.238+0000 7f2b4f7fe640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50102a00 0x7f2b50198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 -- 172.21.15.96:0/318078634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 msgr2=0x7f2b50197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 0x7f2b50197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 -- 172.21.15.96:0/318078634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50102a00 msgr2=0x7f2b50198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50102a00 0x7f2b50198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 -- 172.21.15.96:0/318078634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5019f760 con 0x7f2b501033c0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b54bc4640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800eb10 tx=0x7f2b4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004b20 con 0x7f2b501033c0 2024-03-18T15:39:30.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b48004cc0 con 0x7f2b501033c0 2024-03-18T15:39:30.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48010760 con 0x7f2b501033c0 2024-03-18T15:39:30.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5019fa50 con 0x7f2b501033c0 2024-03-18T15:39:30.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.239+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5019ff20 con 0x7f2b501033c0 2024-03-18T15:39:30.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.240+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b50108ba0 con 0x7f2b501033c0 2024-03-18T15:39:30.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.241+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f2b48005020 con 0x7f2b501033c0 2024-03-18T15:39:30.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.241+0000 7f2b4d7fa640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.242+0000 7f2b4ffff640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.242+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2b480cf1d0 con 0x7f2b501033c0 2024-03-18T15:39:30.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.242+0000 7f2b4ffff640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 0x7f2b380a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b50102860 tx=0x7f2b40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.246+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b4809c0a0 con 0x7f2b501033c0 2024-03-18T15:39:30.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.422+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b500623e0 con 0x7f2b501033c0 2024-03-18T15:39:30.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.423+0000 7f2b4d7fa640 1 -- 172.21.15.96:0/318078634 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7f2b4809b9a0 con 0x7f2b501033c0 2024-03-18T15:39:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:30.424 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 msgr2=0x7f2b380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 0x7f2b380a4d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b50102860 tx=0x7f2b40005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:30.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 msgr2=0x7f2b5019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800eb10 tx=0x7f2b4800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 shutdown_connections 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b380a2890 0x7f2b380a4d50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b501033c0 0x7f2b5019f0f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50102a00 0x7f2b50198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 --2- 172.21.15.96:0/318078634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b50101810 0x7f2b50197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 >> 172.21.15.96:0/318078634 conn(0x7f2b500fcd80 msgr2=0x7f2b50105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 shutdown_connections 2024-03-18T15:39:30.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.425+0000 7f2b56627640 1 -- 172.21.15.96:0/318078634 wait complete. 2024-03-18T15:39:30.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:39:30.438 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:39:30.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- 172.21.15.96:0/1570218559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54101810 msgr2=0x7f7d54101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/1570218559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54101810 0x7f7d54101c30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009900 tx=0x7f7d4402bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- 172.21.15.96:0/1570218559 shutdown_connections 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/1570218559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 0x7f7d54109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/1570218559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54102a00 0x7f7d54102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/1570218559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54101810 0x7f7d54101c30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- 172.21.15.96:0/1570218559 >> 172.21.15.96:0/1570218559 conn(0x7f7d540fcd80 msgr2=0x7f7d540ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- 172.21.15.96:0/1570218559 shutdown_connections 2024-03-18T15:39:30.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- 172.21.15.96:0/1570218559 wait complete. 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 Processor -- start 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.542+0000 7f7d58fe6640 1 -- start start 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54102a00 0x7f7d54198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 0x7f7d5419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d490 con 0x7f7d54101810 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d310 con 0x7f7d541033c0 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5410d610 con 0x7f7d54102a00 2024-03-18T15:39:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36672/0 (socket says 172.21.15.96:36672) 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 -- 172.21.15.96:0/2465120234 learned_addr learned my addr 172.21.15.96:0/2465120234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d51d9b640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54102a00 0x7f7d54198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d52d9d640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 0x7f7d5419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 -- 172.21.15.96:0/2465120234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54102a00 msgr2=0x7f7d54198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54102a00 0x7f7d54198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 -- 172.21.15.96:0/2465120234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 msgr2=0x7f7d5419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 0x7f7d5419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 -- 172.21.15.96:0/2465120234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5419f750 con 0x7f7d54101810 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d5259c640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7d44000c00 tx=0x7f7d4402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44041070 con 0x7f7d54101810 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d44004450 con 0x7f7d54101810 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5419f9e0 con 0x7f7d54101810 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44034440 con 0x7f7d54101810 2024-03-18T15:39:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.543+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5419fed0 con 0x7f7d54101810 2024-03-18T15:39:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.544+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d54108b30 con 0x7f7d54101810 2024-03-18T15:39:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.545+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7d44044070 con 0x7f7d54101810 2024-03-18T15:39:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.546+0000 7f7d437fe640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 0x7f7d3c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.546+0000 7f7d51d9b640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 0x7f7d3c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.546+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7d440f05b0 con 0x7f7d54101810 2024-03-18T15:39:30.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.546+0000 7f7d51d9b640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 0x7f7d3c0a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d54199040 tx=0x7f7d48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.550+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7d440bd030 con 0x7f7d54101810 2024-03-18T15:39:30.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.723+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d5402d3f0 con 0x7f7d54101810 2024-03-18T15:39:30.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.724+0000 7f7d437fe640 1 -- 172.21.15.96:0/2465120234 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f7d440c10d0 con 0x7f7d54101810 2024-03-18T15:39:30.725 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 msgr2=0x7f7d3c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 0x7f7d3c0a4d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d54199040 tx=0x7f7d48005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 msgr2=0x7f7d54197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7d44000c00 tx=0x7f7d4402c640 comp rx=0 tx=0).stop 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 shutdown_connections 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d3c0a2890 0x7f7d3c0a4d50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d541033c0 0x7f7d5419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d54102a00 0x7f7d54198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 --2- 172.21.15.96:0/2465120234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d54101810 0x7f7d54197b30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 >> 172.21.15.96:0/2465120234 conn(0x7f7d540fcd80 msgr2=0x7f7d54105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 shutdown_connections 2024-03-18T15:39:30.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.726+0000 7f7d58fe6640 1 -- 172.21.15.96:0/2465120234 wait complete. 2024-03-18T15:39:30.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:30 smithi096 ceph-mon[31930]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:30.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2410709291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:30.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/978183458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:30.737 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/318078634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:30.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:39:30.818 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:30 smithi193 ceph-mon[36702]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:30.818 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2410709291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:30.818 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/978183458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:30.818 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/318078634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 -- 172.21.15.96:0/3768162701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a41009c0 msgr2=0x7f16a4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 --2- 172.21.15.96:0/3768162701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a41009c0 0x7f16a4100e40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1694009900 tx=0x7f169402bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 -- 172.21.15.96:0/3768162701 shutdown_connections 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 --2- 172.21.15.96:0/3768162701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 0x7f16a4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 --2- 172.21.15.96:0/3768162701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a41009c0 0x7f16a4100e40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 --2- 172.21.15.96:0/3768162701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a4101c90 0x7f16a41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 -- 172.21.15.96:0/3768162701 >> 172.21.15.96:0/3768162701 conn(0x7f16a40fb3d0 msgr2=0x7f16a40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 -- 172.21.15.96:0/3768162701 shutdown_connections 2024-03-18T15:39:30.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.841+0000 7f16a8812640 1 -- 172.21.15.96:0/3768162701 wait complete. 2024-03-18T15:39:30.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 Processor -- start 2024-03-18T15:39:30.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 -- start start 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a41009c0 0x7f16a410b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 0x7f16a41096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d4b0 con 0x7f16a4101c90 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d330 con 0x7f16a4101380 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16a410d630 con 0x7f16a41009c0 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36692/0 (socket says 172.21.15.96:36692) 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 -- 172.21.15.96:0/3233242129 learned_addr learned my addr 172.21.15.96:0/3233242129 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a1d9b640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 0x7f16a41096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a259c640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a41009c0 0x7f16a410b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 -- 172.21.15.96:0/3233242129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a41009c0 msgr2=0x7f16a410b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a41009c0 0x7f16a410b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 -- 172.21.15.96:0/3233242129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 msgr2=0x7f16a41096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 0x7f16a41096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 -- 172.21.15.96:0/3233242129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a410a470 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a2d9d640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f169c00eb10 tx=0x7f169c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c004b20 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169c004cc0 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41a1c30 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.842+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c010760 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.843+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41a2100 con 0x7f16a4101c90 2024-03-18T15:39:30.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.843+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a4068a20 con 0x7f16a4101c90 2024-03-18T15:39:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.844+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f169c005020 con 0x7f16a4101c90 2024-03-18T15:39:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.845+0000 7f16937fe640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 0x7f168c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.845+0000 7f16a259c640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 0x7f168c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.845+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f169c0cf1e0 con 0x7f16a4101c90 2024-03-18T15:39:30.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.845+0000 7f16a259c640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 0x7f168c0a4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f16a4069b30 tx=0x7f1698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:30.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:30.849+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f169c09c110 con 0x7f16a4101c90 2024-03-18T15:39:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:30 smithi137 ceph-mon[35868]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2410709291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/978183458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/318078634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:31.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.023+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f16a402d1e0 con 0x7f16a4101c90 2024-03-18T15:39:31.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.024+0000 7f16937fe640 1 -- 172.21.15.96:0/3233242129 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f169c09ba10 con 0x7f16a4101c90 2024-03-18T15:39:31.025 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 msgr2=0x7f168c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 0x7f168c0a4d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f16a4069b30 tx=0x7f1698005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 msgr2=0x7f16a4109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f169c00eb10 tx=0x7f169c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 shutdown_connections 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f168c0a2890 0x7f168c0a4d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a4101c90 0x7f16a4109be0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a4101380 0x7f16a41096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 --2- 172.21.15.96:0/3233242129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a41009c0 0x7f16a410b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 >> 172.21.15.96:0/3233242129 conn(0x7f16a40fb3d0 msgr2=0x7f16a4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 shutdown_connections 2024-03-18T15:39:31.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.026+0000 7f16a8812640 1 -- 172.21.15.96:0/3233242129 wait complete. 2024-03-18T15:39:31.038 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:39:31.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 -- 172.21.15.96:0/3506518139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2500ffac0 msgr2=0x7fc2500ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 --2- 172.21.15.96:0/3506518139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2500ffac0 0x7fc2500ffee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009930 tx=0x7fc24c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 -- 172.21.15.96:0/3506518139 shutdown_connections 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 --2- 172.21.15.96:0/3506518139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250103450 0x7fc250109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 --2- 172.21.15.96:0/3506518139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc250102a90 0x7fc250102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 --2- 172.21.15.96:0/3506518139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2500ffac0 0x7fc2500ffee0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 -- 172.21.15.96:0/3506518139 >> 172.21.15.96:0/3506518139 conn(0x7fc2500fb3d0 msgr2=0x7fc2500fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 -- 172.21.15.96:0/3506518139 shutdown_connections 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.141+0000 7fc257c4b640 1 -- 172.21.15.96:0/3506518139 wait complete. 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 Processor -- start 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 -- start start 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2500ffac0 0x7fc250195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250102a90 0x7fc250195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010b2f0 con 0x7fc250103450 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010b170 con 0x7fc2500ffac0 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc25010b470 con 0x7fc250102a90 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36706/0 (socket says 172.21.15.96:36706) 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2559e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2500ffac0 0x7fc250195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 -- 172.21.15.96:0/1959763632 learned_addr learned my addr 172.21.15.96:0/1959763632 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2551e6640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250102a90 0x7fc250195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 -- 172.21.15.96:0/1959763632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250102a90 msgr2=0x7fc250195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250102a90 0x7fc250195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 -- 172.21.15.96:0/1959763632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2500ffac0 msgr2=0x7fc250195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2500ffac0 0x7fc250195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 -- 172.21.15.96:0/1959763632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25019d540 con 0x7fc250103450 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc2561e8640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc24800edb0 tx=0x7fc248012490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24801c030 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc248012db0 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248018400 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.142+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25019d830 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.143+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25019dd30 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.143+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc250108bc0 con 0x7fc250103450 2024-03-18T15:39:31.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.144+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7fc24801f070 con 0x7fc250103450 2024-03-18T15:39:31.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.145+0000 7fc246ffd640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 0x7fc23c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.145+0000 7fc2559e7640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 0x7fc23c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.145+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fc2480d2b80 con 0x7fc250103450 2024-03-18T15:39:31.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.145+0000 7fc2559e7640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 0x7fc23c0a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009230 tx=0x7fc24c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.149+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc24809fab0 con 0x7fc250103450 2024-03-18T15:39:31.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.326+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2500623e0 con 0x7fc250103450 2024-03-18T15:39:31.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.326+0000 7fc246ffd640 1 -- 172.21.15.96:0/1959763632 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc24809f3b0 con 0x7fc250103450 2024-03-18T15:39:31.327 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:39:31.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 msgr2=0x7fc23c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 0x7fc23c0a4d50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009230 tx=0x7fc24c02df90 comp rx=0 tx=0).stop 2024-03-18T15:39:31.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 msgr2=0x7fc25019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc24800edb0 tx=0x7fc248012490 comp rx=0 tx=0).stop 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 shutdown_connections 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc23c0a2890 0x7fc23c0a4d50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc250103450 0x7fc25019ced0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc250102a90 0x7fc250195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 --2- 172.21.15.96:0/1959763632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2500ffac0 0x7fc250195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 >> 172.21.15.96:0/1959763632 conn(0x7fc2500fb3d0 msgr2=0x7fc250105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 shutdown_connections 2024-03-18T15:39:31.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.328+0000 7fc257c4b640 1 -- 172.21.15.96:0/1959763632 wait complete. 2024-03-18T15:39:31.340 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:39:31.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1268704796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f4109140 msgr2=0x7f86f410fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1268704796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f4109140 0x7f86f410fb20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002bc40 comp rx=0 tx=0).stop 2024-03-18T15:39:31.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1268704796 shutdown_connections 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1268704796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f4110060 0x7f86f4112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1268704796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f4109140 0x7f86f410fb20 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009900 tx=0x7f86f002bc40 comp rx=0 tx=0).stop 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1268704796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f41087e0 0x7f86f4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.447+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1268704796 >> 172.21.15.96:0/1268704796 conn(0x7f86f406e650 msgr2=0x7f86f4070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.449+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1268704796 shutdown_connections 2024-03-18T15:39:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.450+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1268704796 wait complete. 2024-03-18T15:39:31.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.450+0000 7f86fbe4b640 1 Processor -- start 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.451+0000 7f86fbe4b640 1 -- start start 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f41087e0 0x7f86f41a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41ac090 0x7f86f41ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86f410a7f0 con 0x7f86f41087e0 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86f410a670 con 0x7f86f41ac090 2024-03-18T15:39:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fbe4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f86f410a970 con 0x7f86f4110060 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86f93e6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86f93e6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39218/0 (socket says 172.21.15.96:39218) 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86f93e6640 1 -- 172.21.15.96:0/1900840728 learned_addr learned my addr 172.21.15.96:0/1900840728 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86fa3e8640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41ac090 0x7f86f41ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.452+0000 7f86f9be7640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f41087e0 0x7f86f41a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 -- 172.21.15.96:0/1900840728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41ac090 msgr2=0x7f86f41ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41ac090 0x7f86f41ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 -- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f41087e0 msgr2=0x7f86f41a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f41087e0 0x7f86f41a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 -- 172.21.15.96:0/1900840728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f41aeaf0 con 0x7f86f4110060 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.453+0000 7f86f93e6640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86f002c250 tx=0x7f86f0005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.454+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0041070 con 0x7f86f4110060 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.454+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f41aed80 con 0x7f86f4110060 2024-03-18T15:39:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.454+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f41af190 con 0x7f86f4110060 2024-03-18T15:39:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.454+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f00054e0 con 0x7f86f4110060 2024-03-18T15:39:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.454+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0034440 con 0x7f86f4110060 2024-03-18T15:39:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.458+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f86f0044070 con 0x7f86f4110060 2024-03-18T15:39:31.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.458+0000 7f86eaffd640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 0x7f86e00a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.459+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f86f00f0480 con 0x7f86f4110060 2024-03-18T15:39:31.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.459+0000 7f86f9be7640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 0x7f86e00a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.461+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d80052c0 con 0x7f86f4110060 2024-03-18T15:39:31.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.461+0000 7f86f9be7640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 0x7f86e00a4f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86f4072cd0 tx=0x7f86e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.471+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f86f00bce10 con 0x7f86f4110060 2024-03-18T15:39:31.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2465120234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3233242129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1959763632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.695+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86d8005880 con 0x7f86f4110060 2024-03-18T15:39:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.695+0000 7f86eaffd640 1 -- 172.21.15.96:0/1900840728 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f86f00c0eb0 con 0x7f86f4110060 2024-03-18T15:39:31.696 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:31.696 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:39:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 msgr2=0x7f86e00a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 0x7f86e00a4f20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f86f4072cd0 tx=0x7f86e4005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 msgr2=0x7f86f41a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86f002c250 tx=0x7f86f0005dd0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 shutdown_connections 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f86e00a2a60 0x7f86e00a4f20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41ac090 0x7f86f41ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f86f4110060 0x7f86f41a70a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 --2- 172.21.15.96:0/1900840728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86f41087e0 0x7f86f41a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 >> 172.21.15.96:0/1900840728 conn(0x7f86f406e650 msgr2=0x7f86f406eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 shutdown_connections 2024-03-18T15:39:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.697+0000 7f86fbe4b640 1 -- 172.21.15.96:0/1900840728 wait complete. 2024-03-18T15:39:31.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/329328047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a94101cb0 msgr2=0x7f7a941020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/329328047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a94101cb0 0x7f7a941020d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009900 tx=0x7f7a8402bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/329328047 shutdown_connections 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/329328047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 0x7f7a94108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/329328047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a941009e0 0x7f7a94100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/329328047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a94101cb0 0x7f7a941020d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/329328047 >> 172.21.15.96:0/329328047 conn(0x7f7a940fb410 msgr2=0x7f7a940fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/329328047 shutdown_connections 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/329328047 wait complete. 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 Processor -- start 2024-03-18T15:39:31.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- start start 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 0x7f7a941980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a94101cb0 0x7f7a9419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9410d4d0 con 0x7f7a941009e0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9410d350 con 0x7f7a941013a0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a9bfc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9410d650 con 0x7f7a94101cb0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a99d60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a99d60640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36748/0 (socket says 172.21.15.96:36748) 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.811+0000 7f7a99d60640 1 -- 172.21.15.96:0/1019759784 learned_addr learned my addr 172.21.15.96:0/1019759784 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a9955f640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 0x7f7a941980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a9a561640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a94101cb0 0x7f7a9419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 -- 172.21.15.96:0/1019759784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a94101cb0 msgr2=0x7f7a9419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a94101cb0 0x7f7a9419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 -- 172.21.15.96:0/1019759784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 msgr2=0x7f7a941980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:31.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 0x7f7a941980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 -- 172.21.15.96:0/1019759784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9419f7a0 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a99d60640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a84000c00 tx=0x7f7a8402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84041070 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a84004450 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9419fa30 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84034440 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.812+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9419ff20 con 0x7f7a941009e0 2024-03-18T15:39:31.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.813+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a94107b70 con 0x7f7a941009e0 2024-03-18T15:39:31.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.814+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f7a84044070 con 0x7f7a941009e0 2024-03-18T15:39:31.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.814+0000 7f7a8affd640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 0x7f7a800a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:31.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.815+0000 7f7a9955f640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 0x7f7a800a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:31.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.815+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7a840f05b0 con 0x7f7a941009e0 2024-03-18T15:39:31.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.815+0000 7f7a9955f640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 0x7f7a800a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a94199090 tx=0x7f7a90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:31.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.819+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f7a840bcfa0 con 0x7f7a941009e0 2024-03-18T15:39:31.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2465120234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3233242129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1959763632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2465120234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3233242129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1959763632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:31.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.997+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a940623e0 con 0x7f7a941009e0 2024-03-18T15:39:31.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:31.998+0000 7f7a8affd640 1 -- 172.21.15.96:0/1019759784 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7f7a840c1040 con 0x7f7a941009e0 2024-03-18T15:39:31.999 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:31.999 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 msgr2=0x7f7a800a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 0x7f7a800a4d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a94199090 tx=0x7f7a90005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 msgr2=0x7f7a94197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a84000c00 tx=0x7f7a8402c650 comp rx=0 tx=0).stop 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 shutdown_connections 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7a800a2890 0x7f7a800a4d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a94101cb0 0x7f7a9419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a941013a0 0x7f7a941980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 --2- 172.21.15.96:0/1019759784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a941009e0 0x7f7a94197b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 >> 172.21.15.96:0/1019759784 conn(0x7f7a940fb410 msgr2=0x7f7a94104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 shutdown_connections 2024-03-18T15:39:32.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.000+0000 7f7a9bfc4640 1 -- 172.21.15.96:0/1019759784 wait complete. 2024-03-18T15:39:32.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:39:32.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 -- 172.21.15.96:0/1808973746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 msgr2=0x7f82341082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 --2- 172.21.15.96:0/1808973746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f82341082c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f822c009ca0 tx=0x7f822c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 -- 172.21.15.96:0/1808973746 shutdown_connections 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 --2- 172.21.15.96:0/1808973746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f82341082c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 --2- 172.21.15.96:0/1808973746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 0x7f8234101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 --2- 172.21.15.96:0/1808973746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f82341002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 -- 172.21.15.96:0/1808973746 >> 172.21.15.96:0/1808973746 conn(0x7f82340fb400 msgr2=0x7f82340fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 -- 172.21.15.96:0/1808973746 shutdown_connections 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 -- 172.21.15.96:0/1808973746 wait complete. 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.113+0000 7f8239c6c640 1 Processor -- start 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- start start 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f8234197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 0x7f8234198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f823410b290 con 0x7f82341019a0 2024-03-18T15:39:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f823410b110 con 0x7f8234100fe0 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f823410b410 con 0x7f82340ffe90 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36754/0 (socket says 172.21.15.96:36754) 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 -- 172.21.15.96:0/2933069707 learned_addr learned my addr 172.21.15.96:0/2933069707 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f822bfff640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 0x7f8234198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f82337fe640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f8234197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 -- 172.21.15.96:0/2933069707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 msgr2=0x7f8234197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f8234197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 -- 172.21.15.96:0/2933069707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 msgr2=0x7f8234198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 0x7f8234198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 -- 172.21.15.96:0/2933069707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823419f760 con 0x7f82341019a0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f82337fe640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f8234197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8233fff640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f822c037040 tx=0x7f822c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c041020 con 0x7f82341019a0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822c034cd0 con 0x7f82341019a0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c039040 con 0x7f82341019a0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823419f9f0 con 0x7f82341019a0 2024-03-18T15:39:32.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.114+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823419fee0 con 0x7f82341019a0 2024-03-18T15:39:32.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.115+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82341a01d0 con 0x7f82341019a0 2024-03-18T15:39:32.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.117+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f822c044020 con 0x7f82341019a0 2024-03-18T15:39:32.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.118+0000 7f82317fa640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 0x7f82200a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.118+0000 7f82337fe640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 0x7f82200a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.118+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f822c0f0e00 con 0x7f82341019a0 2024-03-18T15:39:32.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.118+0000 7f82337fe640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 0x7f82200a4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8224009c20 tx=0x7f8224005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.121+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f822c04b020 con 0x7f82341019a0 2024-03-18T15:39:32.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.337+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82341a0560 con 0x7f82341019a0 2024-03-18T15:39:32.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.337+0000 7f82317fa640 1 -- 172.21.15.96:0/2933069707 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f822c0bded0 con 0x7f82341019a0 2024-03-18T15:39:32.339 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:32.339 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:39:32.339 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:32.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 msgr2=0x7f82200a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 0x7f82200a4d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8224009c20 tx=0x7f8224005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:32.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 msgr2=0x7f823419f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f822c037040 tx=0x7f822c032f40 comp rx=0 tx=0).stop 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 shutdown_connections 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f82200a2890 0x7f82200a4d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82341019a0 0x7f823419f0f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8234100fe0 0x7f8234198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 --2- 172.21.15.96:0/2933069707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82340ffe90 0x7f8234197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 >> 172.21.15.96:0/2933069707 conn(0x7f82340fb400 msgr2=0x7f82341064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.339+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 shutdown_connections 2024-03-18T15:39:32.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.340+0000 7f8239c6c640 1 -- 172.21.15.96:0/2933069707 wait complete. 2024-03-18T15:39:32.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T15:38:04.167866+0000', 'modified': '2024-03-18T15:38:11.142722+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24493, 'mds_1': 24485, 'mds_2': 24472, 'mds_3': 24460, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/36196340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 36196340}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 36196340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/3021384858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 3021384858}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 3021384858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/753223976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 753223976}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 753223976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3317488579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3317488579}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3317488579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3504042144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3504042144}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3504042144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6837/2776916439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 2776916439}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 2776916439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.193:6835/1114653826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1114653826}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1114653826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6837/2478672579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2478672579}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2478672579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/1070718965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1070718965}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1070718965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.193:6839/1184009925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 1184009925}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 1184009925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24460, 24452, 24485, 24472, 24493]} 2024-03-18T15:39:32.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T15:39:32.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 -- 172.21.15.96:0/219075481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c100960 msgr2=0x7f6c0c100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 --2- 172.21.15.96:0/219075481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c100960 0x7f6c0c100de0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8009900 tx=0x7f6bf802bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 -- 172.21.15.96:0/219075481 shutdown_connections 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 --2- 172.21.15.96:0/219075481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101320 0x7f6c0c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 --2- 172.21.15.96:0/219075481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c100960 0x7f6c0c100de0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 --2- 172.21.15.96:0/219075481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c101d10 0x7f6c0c102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 -- 172.21.15.96:0/219075481 >> 172.21.15.96:0/219075481 conn(0x7f6c0c0fb410 msgr2=0x7f6c0c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 -- 172.21.15.96:0/219075481 shutdown_connections 2024-03-18T15:39:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.454+0000 7f6c130bf640 1 -- 172.21.15.96:0/219075481 wait complete. 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 Processor -- start 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 -- start start 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 0x7f6c0c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 0x7f6c0c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c10c3a0 con 0x7f6c0c101320 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c10c220 con 0x7f6c0c100960 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c10c520 con 0x7f6c0c101d10 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36782/0 (socket says 172.21.15.96:36782) 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 -- 172.21.15.96:0/1921624722 learned_addr learned my addr 172.21.15.96:0/1921624722 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 0x7f6c0c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c1165c640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 0x7f6c0c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 -- 172.21.15.96:0/1921624722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 msgr2=0x7f6c0c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 0x7f6c0c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 -- 172.21.15.96:0/1921624722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 msgr2=0x7f6c0c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 0x7f6c0c1003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 -- 172.21.15.96:0/1921624722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c19f740 con 0x7f6c0c101320 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 0x7f6c0c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c1165c640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 0x7f6c0c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8000c00 tx=0x7f6bf802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c09ffb640 1 -- 172.21.15.96:0/1921624722 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8041070 con 0x7f6c0c101320 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c09ffb640 1 -- 172.21.15.96:0/1921624722 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf8004450 con 0x7f6c0c101320 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.455+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c19f9d0 con 0x7f6c0c101320 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.456+0000 7f6c09ffb640 1 -- 172.21.15.96:0/1921624722 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8034440 con 0x7f6c0c101320 2024-03-18T15:39:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.456+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c19fec0 con 0x7f6c0c101320 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c09ffb640 1 -- 172.21.15.96:0/1921624722 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 137341+0+0 (secure 0 0 0) 0x7f6bf8044070 con 0x7f6c0c101320 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 Processor -- start 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 -- start start 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c09ffb640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 0x7f6bfc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5257cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d524d8c80 con 0x562d52574bd0 2024-03-18T15:39:32.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d52481aa0 con 0x562d5257a730 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c15d82740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d524d4d20 con 0x562d525755c0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.458+0000 7f6c1165c640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 0x7f6bfc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c09ffb640 1 -- 172.21.15.96:0/1921624722 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6bf80f0790 con 0x7f6c0c101320 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36796/0 (socket says 172.21.15.96:36796) 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 -- 172.21.15.96:0/1705176265 learned_addr learned my addr 172.21.15.96:0/1705176265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c1165c640 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5257cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 -- 172.21.15.96:0/1705176265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 msgr2=0x562d5257a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 -- 172.21.15.96:0/1705176265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 msgr2=0x562d5257cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5257cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 -- 172.21.15.96:0/1705176265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d524a3ff0 con 0x562d52574bd0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c1165c640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 0x7f6bfc0a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c04009eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009cd0 tx=0x7f6c000117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5498bb391bf5a6d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6bf6ffd640 1 -- 172.21.15.96:0/1705176265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00012450 con 0x562d52574bd0 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6bf6ffd640 1 -- 172.21.15.96:0/1705176265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c00012ad0 con 0x562d52574bd0 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.459+0000 7f6bf6ffd640 1 -- 172.21.15.96:0/1705176265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000fc50 con 0x562d52574bd0 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- 172.21.15.96:0/1705176265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 msgr2=0x562d52574ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009cd0 tx=0x7f6c000117b0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- 172.21.15.96:0/1705176265 shutdown_connections 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5257cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 --2- 172.21.15.96:0/1705176265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52574ff0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- 172.21.15.96:0/1705176265 >> 172.21.15.96:0/1705176265 conn(0x562d52570140 msgr2=0x562d52570540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- 172.21.15.96:0/1705176265 shutdown_connections 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- 172.21.15.96:0/1705176265 wait complete. 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 -- start start 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.460+0000 7f6c15d82740 1 Processor -- start 2024-03-18T15:39:32.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 0x562d5257ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d525862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d5257d6e0 con 0x562d525755c0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d52494ef0 con 0x562d5257a730 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d5257d860 con 0x562d52574bd0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 10 client.0 _ll_register_callbacks cb 0x562d5248fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 0x562d5257ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36812/0 (socket says 172.21.15.96:36812) 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 -- 172.21.15.96:0/3823741805 learned_addr learned my addr 172.21.15.96:0/3823741805 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c1165c640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d525862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 -- 172.21.15.96:0/3823741805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 msgr2=0x562d5257ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 0x562d5257ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 -- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 msgr2=0x562d525862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d525862d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 -- 172.21.15.96:0/3823741805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d524a1a30 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 0x562d5257ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80b9aa0 tx=0x7f6bf80bcc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75091aa3fd455fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf804b040 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6bf8042d80 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c00012ad0 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6bf8004c80 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.461+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562d521ed2d0 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c00012ad0 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6bf80035f0 con 0x562d525755c0 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 20 client.15066 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d5258b870 0x562d5258cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 4 client.15066 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d524e61c0 con 0x562d5258b870 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d5258b870 0x562d5258cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6bebfff640 10 client.15066 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.462+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d5258b870 0x562d5258cce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d553d52910530429 server_cookie=e3c2d88c8b2639e0 in_seq=0 out_seq=0 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6bebfff640 1 -- 172.21.15.96:0/3823741805 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d524e61c0 con 0x562d5258b870 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6bebfff640 10 client.15066 handle_command_reply: tid=0 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 client.15066 shutdown 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 msgr2=0x562d5257f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80b9aa0 tx=0x7f6bf80bcc10 comp rx=0 tx=0).stop 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 shutdown_connections 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d5258b870 0x562d5258cce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d525862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d525755c0 0x562d5257f290 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 --2- 172.21.15.96:0/3823741805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d52574bd0 0x562d5257ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6bebfff640 0 client.15066 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:32.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 >> 172.21.15.96:0/3823741805 conn(0x562d52570140 msgr2=0x562d5256f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 shutdown_connections 2024-03-18T15:39:32.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 1 -- 172.21.15.96:0/3823741805 wait complete. 2024-03-18T15:39:32.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.463+0000 7f6c15d82740 20 client.15066 trim_cache size 0 max 16384 2024-03-18T15:39:32.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.481+0000 7f6c15d82740 1 Processor -- start 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 -- start start 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5256eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d5258b020 con 0x562d52574bd0 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d5258b6f0 con 0x562d5257a730 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c15d82740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d5258bdc0 con 0x562d525755c0 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36820/0 (socket says 172.21.15.96:36820) 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 -- 172.21.15.96:0/528247418 learned_addr learned my addr 172.21.15.96:0/528247418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c1165c640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5256eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 -- 172.21.15.96:0/528247418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 msgr2=0x562d5257f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 -- 172.21.15.96:0/528247418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 msgr2=0x562d5256eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5256eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 -- 172.21.15.96:0/528247418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d522b76b0 con 0x562d52574bd0 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c1165c640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5256eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c00000c00 tx=0x7f6c00012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a42d81e6bf025a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6bf67fc640 1 -- 172.21.15.96:0/528247418 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00009530 con 0x562d52574bd0 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6bf67fc640 1 -- 172.21.15.96:0/528247418 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c00030030 con 0x562d52574bd0 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.482+0000 7f6bf67fc640 1 -- 172.21.15.96:0/528247418 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00012450 con 0x562d52574bd0 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- 172.21.15.96:0/528247418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 msgr2=0x562d52570520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c00000c00 tx=0x7f6c00012f90 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- 172.21.15.96:0/528247418 shutdown_connections 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5257a730 0x562d5256eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d525755c0 0x562d5257f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 --2- 172.21.15.96:0/528247418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d52570520 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- 172.21.15.96:0/528247418 >> 172.21.15.96:0/528247418 conn(0x562d52570140 msgr2=0x562d5257e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- 172.21.15.96:0/528247418 shutdown_connections 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- 172.21.15.96:0/528247418 wait complete. 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 -- start start 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.483+0000 7f6c15d82740 1 Processor -- start 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d525755c0 0x562d52585d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d5257a730 0x562d525f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d5258cb60 con 0x562d52574bd0 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d5258c490 con 0x562d525755c0 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d5258d230 con 0x562d5257a730 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 10 client.0 _ll_register_callbacks cb 0x7f6bf80b9370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36824/0 (socket says 172.21.15.96:36824) 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 -- 172.21.15.96:0/2024974993 learned_addr learned my addr 172.21.15.96:0/2024974993 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d525755c0 0x562d52585d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c1165c640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d5257a730 0x562d525f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 -- 172.21.15.96:0/2024974993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d5257a730 msgr2=0x562d525f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d5257a730 0x562d525f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 -- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d525755c0 msgr2=0x562d52585d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d525755c0 0x562d52585d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 -- 172.21.15.96:0/2024974993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d525f4020 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c10e5b640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c000099f0 tx=0x7f6c00009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9e3f8634424c91 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00039070 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6c000050b0 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.484+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c00030030 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6c00044050 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 10 client.15072 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562d525f5320 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c00030030 con 0x562d52574bd0 2024-03-18T15:39:32.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6c000035f0 con 0x562d52574bd0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 20 client.15072 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d525f98c0 0x562d525f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 4 client.15072 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "status"}) -- 0x562d524e0510 con 0x562d525f98c0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d525f98c0 0x562d525f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6bebfff640 10 client.15072 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.485+0000 7f6c0bfff640 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d525f98c0 0x562d525f85d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65976bb059e1e153 server_cookie=7622a24edf131942 in_seq=0 out_seq=0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6bebfff640 1 -- 172.21.15.96:0/2024974993 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x562d524e0510 con 0x562d525f98c0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6bebfff640 10 client.15072 handle_command_reply: tid=0 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 client.15072 shutdown 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 msgr2=0x562d525857c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c000099f0 tx=0x7f6c00009310 comp rx=0 tx=0).stop 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 shutdown_connections 2024-03-18T15:39:32.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562d525f98c0 0x562d525f85d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d5257a730 0x562d525f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6bebfff640 0 client.15072 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d525755c0 0x562d52585d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 --2- 172.21.15.96:0/2024974993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d52574bd0 0x562d525857c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 >> 172.21.15.96:0/2024974993 conn(0x562d52570140 msgr2=0x562d525733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 shutdown_connections 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 1 -- 172.21.15.96:0/2024974993 wait complete. 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.486+0000 7f6c15d82740 20 client.15072 trim_cache size 0 max 16384 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-03-18T15:39:32.493 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24452, 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 82.295681152, 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 105.928945145 2024-03-18T15:39:32.494 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:32.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 msgr2=0x7f6bfc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 0x7f6bfc0a4d50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c04009eb0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 msgr2=0x7f6c0c19c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8000c00 tx=0x7f6bf802c650 comp rx=0 tx=0).stop 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 shutdown_connections 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6bfc0a2890 0x7f6bfc0a4d50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c0c101d10 0x7f6c0c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c0c101320 0x7f6c0c19c840 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 --2- 172.21.15.96:0/1921624722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c0c100960 0x7f6c0c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 >> 172.21.15.96:0/1921624722 conn(0x7f6c0c0fb410 msgr2=0x7f6c0c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 shutdown_connections 2024-03-18T15:39:32.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.487+0000 7f6c130bf640 1 -- 172.21.15.96:0/1921624722 wait complete. 2024-03-18T15:39:32.498 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 -- 172.21.15.96:0/119098149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 msgr2=0x7f6d14100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/119098149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14100b10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009900 tx=0x7f6d1002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 -- 172.21.15.96:0/119098149 shutdown_connections 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/119098149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 0x7f6d14108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/119098149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 0x7f6d14101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/119098149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14100b10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 -- 172.21.15.96:0/119098149 >> 172.21.15.96:0/119098149 conn(0x7f6d140fbc80 msgr2=0x7f6d140fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 -- 172.21.15.96:0/119098149 shutdown_connections 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.604+0000 7f6d1c70c640 1 -- 172.21.15.96:0/119098149 wait complete. 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 Processor -- start 2024-03-18T15:39:32.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 -- start start 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 0x7f6d14196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 0x7f6d1419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410c360 con 0x7f6d141006f0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410c1e0 con 0x7f6d141022a0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1410c4e0 con 0x7f6d141018e0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36848/0 (socket says 172.21.15.96:36848) 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/2058878085 learned_addr learned my addr 172.21.15.96:0/2058878085 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 0x7f6d14196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 0x7f6d1419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/2058878085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 msgr2=0x7f6d14196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 0x7f6d14196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/2058878085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 msgr2=0x7f6d1419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 0x7f6d1419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/2058878085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1419e650 con 0x7f6d141006f0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d10000c00 tx=0x7f6d1002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d0b7fe640 1 -- 172.21.15.96:0/2058878085 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10041070 con 0x7f6d141006f0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1419e8e0 con 0x7f6d141006f0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d0b7fe640 1 -- 172.21.15.96:0/2058878085 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10004450 con 0x7f6d141006f0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.605+0000 7f6d0b7fe640 1 -- 172.21.15.96:0/2058878085 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10034440 con 0x7f6d141006f0 2024-03-18T15:39:32.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.606+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d14069fb0 con 0x7f6d141006f0 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.608+0000 7f6d0b7fe640 1 -- 172.21.15.96:0/2058878085 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f6d10044070 con 0x7f6d141006f0 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.608+0000 7f6d1f3cc740 1 Processor -- start 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.608+0000 7f6d1f3cc740 1 -- start start 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.608+0000 7f6d0b7fe640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 0x7f6d000a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.608+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac859bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac7b5c80 con 0x55d2ac851bd0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac75eaa0 con 0x55d2ac857730 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac7b1d20 con 0x55d2ac8525c0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 0x7f6d000a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39334/0 (socket says 172.21.15.96:39334) 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d0b7fe640 1 -- 172.21.15.96:0/2058878085 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d100f0530 con 0x7f6d141006f0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36864/0 (socket says 172.21.15.96:36864) 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 -- 172.21.15.96:0/4129822794 learned_addr learned my addr 172.21.15.96:0/4129822794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac859bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 -- 172.21.15.96:0/4129822794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 msgr2=0x55d2ac859bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac859bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 -- 172.21.15.96:0/4129822794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 msgr2=0x55d2ac851ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 -- 172.21.15.96:0/4129822794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ac780ff0 con 0x55d2ac8525c0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 0x7f6d000a6ed0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c005fd0 tx=0x7f6d0c009e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d04013c70 tx=0x7f6d04009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30d1cabd778c083 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d08ff9640 1 -- 172.21.15.96:0/4129822794 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04021070 con 0x55d2ac8525c0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d08ff9640 1 -- 172.21.15.96:0/4129822794 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d0400edf0 con 0x55d2ac8525c0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.609+0000 7f6d08ff9640 1 -- 172.21.15.96:0/4129822794 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0401cca0 con 0x55d2ac8525c0 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/4129822794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 msgr2=0x55d2ac8570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d04013c70 tx=0x7f6d04009b50 comp rx=0 tx=0).stop 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/4129822794 shutdown_connections 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac859bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac8570c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/4129822794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac851ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/4129822794 >> 172.21.15.96:0/4129822794 conn(0x55d2ac84d140 msgr2=0x55d2ac84d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/4129822794 shutdown_connections 2024-03-18T15:39:32.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/4129822794 wait complete. 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.610+0000 7f6d1f3cc740 1 -- start start 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 Processor -- start 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac85a6e0 con 0x55d2ac857730 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac771ef0 con 0x55d2ac851bd0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac85a860 con 0x55d2ac8525c0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 10 client.0 _ll_register_callbacks cb 0x55d2ac76cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36868/0 (socket says 172.21.15.96:36868) 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d19ca7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 -- 172.21.15.96:0/1782662841 learned_addr learned my addr 172.21.15.96:0/1782662841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 -- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 msgr2=0x55d2ac85c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 -- 172.21.15.96:0/1782662841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac851bd0 msgr2=0x55d2ac85bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 -- 172.21.15.96:0/1782662841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ac77ea30 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c013390 tx=0x7f6d0c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906bafd75a1e7a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c021410 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d0c0215b0 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.611+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0400edf0 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d0c0217b0 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2ac4ca2d0 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d0400edf0 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6d0c034050 con 0x55d2ac857730 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 10 client.15084 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 20 client.15084 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac868850 0x55d2ac869cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 4 client.15084 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2ac7c31c0 con 0x55d2ac868850 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.612+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac868850 0x55d2ac869cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.613+0000 7f6cfdffb640 10 client.15084 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.613+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac868850 0x55d2ac869cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45ebe72aed7d21c0 server_cookie=dbf4b7df4ed7f737 in_seq=0 out_seq=0 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6cfdffb640 1 -- 172.21.15.96:0/1782662841 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d2ac7c31c0 con 0x55d2ac868850 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6cfdffb640 10 client.15084 handle_command_reply: tid=0 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 client.15084 shutdown 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 msgr2=0x55d2ac863320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c013390 tx=0x7f6d0c004ab0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 shutdown_connections 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac868850 0x55d2ac869cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac857730 0x55d2ac863320 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6cfdffb640 0 client.15084 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/1782662841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 >> 172.21.15.96:0/1782662841 conn(0x55d2ac84d140 msgr2=0x55d2ac84c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 shutdown_connections 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/1782662841 wait complete. 2024-03-18T15:39:32.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.614+0000 7f6d1f3cc740 20 client.15084 trim_cache size 0 max 16384 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.631+0000 7f6d1f3cc740 1 Processor -- start 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.631+0000 7f6d1f3cc740 1 -- start start 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac863240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac868000 con 0x55d2ac851bd0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac8686d0 con 0x55d2ac857730 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac868da0 con 0x55d2ac8525c0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36884/0 (socket says 172.21.15.96:36884) 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/660412239 learned_addr learned my addr 172.21.15.96:0/660412239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac863240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/660412239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 msgr2=0x55d2ac85c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/660412239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 msgr2=0x55d2ac863240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac863240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 -- 172.21.15.96:0/660412239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ac5946b0 con 0x55d2ac851bd0 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac863240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6d100b99d0 tx=0x7f6d100edef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e890732327f3af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6cfffff640 1 -- 172.21.15.96:0/660412239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1004b040 con 0x55d2ac851bd0 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6cfffff640 1 -- 172.21.15.96:0/660412239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d100cf690 con 0x55d2ac851bd0 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.632+0000 7f6cfffff640 1 -- 172.21.15.96:0/660412239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100bb130 con 0x55d2ac851bd0 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/660412239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 msgr2=0x55d2ac85bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6d100b99d0 tx=0x7f6d100edef0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/660412239 shutdown_connections 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac863240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac8525c0 0x55d2ac85c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/660412239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac851bd0 0x55d2ac85bda0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/660412239 >> 172.21.15.96:0/660412239 conn(0x55d2ac84d140 msgr2=0x55d2ac856de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/660412239 shutdown_connections 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/660412239 wait complete. 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- start start 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 Processor -- start 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac851bd0 0x55d2ac861230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac8ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac869b40 con 0x55d2ac8525c0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac869470 con 0x55d2ac857730 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2ac86a210 con 0x55d2ac851bd0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1f3cc740 10 client.0 _ll_register_callbacks cb 0x7f6d0c0135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d19ca7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d19ca7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36886/0 (socket says 172.21.15.96:36886) 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d19ca7640 1 -- 172.21.15.96:0/2308727613 learned_addr learned my addr 172.21.15.96:0/2308727613 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.633+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac851bd0 0x55d2ac861230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1aca9640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac8ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 -- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac851bd0 msgr2=0x55d2ac861230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac851bd0 0x55d2ac861230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 -- 172.21.15.96:0/2308727613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 msgr2=0x55d2ac8ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac8ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 -- 172.21.15.96:0/2308727613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ac8cd030 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d19ca7640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e780 tx=0x7f6d04013940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56ef1dbb7d94b7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0402b020 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f6d0400eb20 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d100cf690 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6d04036050 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d2ac8ce2d0 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d100cf690 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6d04003920 con 0x55d2ac8525c0 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 10 client.15090 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 20 client.15090 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac8d2ce0 0x55d2ac8d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 4 client.15090 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.634+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "status"}) -- 0x55d2ac7bd510 con 0x55d2ac8d2ce0 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac8d2ce0 0x55d2ac8d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6cfdffb640 10 client.15090 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1a4a8640 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac8d2ce0 0x55d2ac8d19f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db3292f03dd2fc5a server_cookie=a888117cd84a7d8c in_seq=0 out_seq=0 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6cfdffb640 1 -- 172.21.15.96:0/2308727613 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d2ac7bd510 con 0x55d2ac8d2ce0 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6cfdffb640 10 client.15090 handle_command_reply: tid=0 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 client.15090 shutdown 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 msgr2=0x55d2ac8ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d0400e780 tx=0x7f6d04013940 comp rx=0 tx=0).stop 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 shutdown_connections 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55d2ac8d2ce0 0x55d2ac8d19f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2ac857730 0x55d2ac8ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2ac8525c0 0x55d2ac8ca5a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 --2- 172.21.15.96:0/2308727613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2ac851bd0 0x55d2ac861230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6cfdffb640 0 client.15090 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 >> 172.21.15.96:0/2308727613 conn(0x55d2ac84d140 msgr2=0x55d2ac867610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 shutdown_connections 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 1 -- 172.21.15.96:0/2308727613 wait complete. 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.635+0000 7f6d1f3cc740 20 client.15090 trim_cache size 0 max 16384 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24460, 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:32.643 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 83.464891258999998, 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 111.72929994099999 2024-03-18T15:39:32.644 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 msgr2=0x7f6d000a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 0x7f6d000a6ed0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c005fd0 tx=0x7f6d0c009e60 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 msgr2=0x7f6d14196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d10000c00 tx=0x7f6d1002c640 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 shutdown_connections 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6d000a4a10 0x7f6d000a6ed0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d141022a0 0x7f6d1419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d141018e0 0x7f6d14196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 --2- 172.21.15.96:0/2058878085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d141006f0 0x7f6d14196a30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 >> 172.21.15.96:0/2058878085 conn(0x7f6d140fbc80 msgr2=0x7f6d14104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 shutdown_connections 2024-03-18T15:39:32.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.636+0000 7f6d1c70c640 1 -- 172.21.15.96:0/2058878085 wait complete. 2024-03-18T15:39:32.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:32 smithi096 ceph-mon[31930]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:32.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1900840728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1019759784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2933069707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-18T15:39:32.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 -- 172.21.15.96:0/2223721098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 msgr2=0x7faf0c109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 --2- 172.21.15.96:0/2223721098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c109ce0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faf04009c10 tx=0x7faf0402bef0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 -- 172.21.15.96:0/2223721098 shutdown_connections 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 --2- 172.21.15.96:0/2223721098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c109ce0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 --2- 172.21.15.96:0/2223721098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c102a90 0x7faf0c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 --2- 172.21.15.96:0/2223721098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c0ffac0 0x7faf0c0ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 -- 172.21.15.96:0/2223721098 >> 172.21.15.96:0/2223721098 conn(0x7faf0c0fb3f0 msgr2=0x7faf0c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.751+0000 7faf12834640 1 -- 172.21.15.96:0/2223721098 shutdown_connections 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 -- 172.21.15.96:0/2223721098 wait complete. 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 Processor -- start 2024-03-18T15:39:32.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 -- start start 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c0ffac0 0x7faf0c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d4a0 con 0x7faf0c102a90 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d320 con 0x7faf0c0ffac0 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf12834640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10d620 con 0x7faf0c103450 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36910/0 (socket says 172.21.15.96:36910) 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf0b7fe640 1 -- 172.21.15.96:0/2560984054 learned_addr learned my addr 172.21.15.96:0/2560984054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf0bfff640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c0ffac0 0x7faf0c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.752+0000 7faf10dd1640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 -- 172.21.15.96:0/2560984054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 msgr2=0x7faf0c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 -- 172.21.15.96:0/2560984054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c0ffac0 msgr2=0x7faf0c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c0ffac0 0x7faf0c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 -- 172.21.15.96:0/2560984054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c19f6e0 con 0x7faf0c102a90 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e9f0 tx=0x7faf0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf097fa640 1 -- 172.21.15.96:0/2560984054 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000cd80 con 0x7faf0c102a90 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c19f9d0 con 0x7faf0c102a90 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf097fa640 1 -- 172.21.15.96:0/2560984054 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf00005500 con 0x7faf0c102a90 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf097fa640 1 -- 172.21.15.96:0/2560984054 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00010690 con 0x7faf0c102a90 2024-03-18T15:39:32.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.753+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c19ff20 con 0x7faf0c102a90 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.755+0000 7faf097fa640 1 -- 172.21.15.96:0/2560984054 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7faf000046c0 con 0x7faf0c102a90 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.755+0000 7faf15419740 1 Processor -- start 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.755+0000 7faf15419740 1 -- start start 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf097fa640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822eab40 0x5556822ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555682244c80 con 0x5556822e59d0 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556821edaa0 con 0x5556822e4fe0 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf15419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555682240d20 con 0x5556822eab40 2024-03-18T15:39:32.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf10dd1640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36916/0 (socket says 172.21.15.96:36916) 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 -- 172.21.15.96:0/4161261593 learned_addr learned my addr 172.21.15.96:0/4161261593 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf10dd1640 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822eab40 0x5556822ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf097fa640 1 -- 172.21.15.96:0/2560984054 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faf000cee60 con 0x7faf0c102a90 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 -- 172.21.15.96:0/4161261593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822eab40 msgr2=0x5556822ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822eab40 0x5556822ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 -- 172.21.15.96:0/4161261593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 msgr2=0x5556822e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 -- 172.21.15.96:0/4161261593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568220fff0 con 0x5556822e59d0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0bfff640 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf10dd1640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 0x7faef40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf04006010 tx=0x7faf040098d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf000cccc0 tx=0x7faf00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498eba7accafe8f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faef27fc640 1 -- 172.21.15.96:0/4161261593 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0009b330 con 0x5556822e59d0 2024-03-18T15:39:32.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faef27fc640 1 -- 172.21.15.96:0/4161261593 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0009bef0 con 0x5556822e59d0 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.756+0000 7faef27fc640 1 -- 172.21.15.96:0/4161261593 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf000b48b0 con 0x5556822e59d0 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- 172.21.15.96:0/4161261593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 msgr2=0x5556822ea4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf000cccc0 tx=0x7faf00004a40 comp rx=0 tx=0).stop 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- 172.21.15.96:0/4161261593 shutdown_connections 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822eab40 0x5556822ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e59d0 0x5556822ea4d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 --2- 172.21.15.96:0/4161261593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- 172.21.15.96:0/4161261593 >> 172.21.15.96:0/4161261593 conn(0x5556822e0550 msgr2=0x5556822e0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- 172.21.15.96:0/4161261593 shutdown_connections 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- 172.21.15.96:0/4161261593 wait complete. 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.757+0000 7faf15419740 1 -- start start 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 Processor -- start 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e4fe0 0x5556822ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e59d0 0x5556822ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556822edaf0 con 0x5556822eab40 2024-03-18T15:39:32.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555682200ef0 con 0x5556822e59d0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5556822edc70 con 0x5556822e4fe0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf15419740 10 client.0 _ll_register_callbacks cb 0x5556821fbdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36926/0 (socket says 172.21.15.96:36926) 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 -- 172.21.15.96:0/3590033567 learned_addr learned my addr 172.21.15.96:0/3590033567 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e4fe0 0x5556822ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e59d0 0x5556822ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 -- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e4fe0 msgr2=0x5556822ef1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e4fe0 0x5556822ef1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 -- 172.21.15.96:0/3590033567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e59d0 msgr2=0x5556822ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e59d0 0x5556822ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 -- 172.21.15.96:0/3590033567 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568220da30 con 0x5556822eab40 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faf04035000 tx=0x7faf0402bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2c4676b713592 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.758+0000 7faeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf040336a0 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faf0403a040 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0009bef0 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf0402cd90 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555681f592d0 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf0009bef0 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faf04034430 con 0x5556822eab40 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 20 client.15102 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5556822fbd80 0x5556822fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 4 client.15102 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556822521c0 con 0x5556822fbd80 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.759+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5556822fbd80 0x5556822fd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.760+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5556822fbd80 0x5556822fd1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56e5e27cc15acede server_cookie=861511cb8a349c67 in_seq=0 out_seq=0 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.760+0000 7faeeb7fe640 10 client.15102 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:32.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.760+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3590033567 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5556822521c0 con 0x5556822fbd80 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.760+0000 7faeeb7fe640 10 client.15102 handle_command_reply: tid=0 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.760+0000 7faf15419740 1 client.15102 shutdown 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 msgr2=0x5556822f6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faf04035000 tx=0x7faf0402bef0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 shutdown_connections 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5556822fbd80 0x5556822fd1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822f6770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e59d0 0x5556822ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 --2- 172.21.15.96:0/3590033567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e4fe0 0x5556822ef1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faeeb7fe640 0 client.15102 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 >> 172.21.15.96:0/3590033567 conn(0x5556822e0550 msgr2=0x5556822db380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 shutdown_connections 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 1 -- 172.21.15.96:0/3590033567 wait complete. 2024-03-18T15:39:32.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.761+0000 7faf15419740 20 client.15102 trim_cache size 0 max 16384 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 Processor -- start 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 -- start start 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556822fb530 con 0x5556822eab40 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556822fbc00 con 0x5556822e4fe0 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf15419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5556822fc2d0 con 0x5556822e59d0 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf10dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf10dd1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36940/0 (socket says 172.21.15.96:36940) 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf10dd1640 1 -- 172.21.15.96:0/3757966621 learned_addr learned my addr 172.21.15.96:0/3757966621 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.778+0000 7faf10dd1640 1 -- 172.21.15.96:0/3757966621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 msgr2=0x5556822f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf10dd1640 1 -- 172.21.15.96:0/3757966621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 msgr2=0x5556822fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf10dd1640 1 -- 172.21.15.96:0/3757966621 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556820236b0 con 0x5556822eab40 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faf0402c5e0 tx=0x7faf04005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7b5ee8a73c10d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faef1ffb640 1 -- 172.21.15.96:0/3757966621 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04033bd0 con 0x5556822eab40 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faef1ffb640 1 -- 172.21.15.96:0/3757966621 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf04053070 con 0x5556822eab40 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faef1ffb640 1 -- 172.21.15.96:0/3757966621 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04034690 con 0x5556822eab40 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 -- 172.21.15.96:0/3757966621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 msgr2=0x5556822ea180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faf0402c5e0 tx=0x7faf04005d10 comp rx=0 tx=0).stop 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 -- 172.21.15.96:0/3757966621 shutdown_connections 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822eab40 0x5556822ea180 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 --2- 172.21.15.96:0/3757966621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822e4fe0 0x5556822fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 -- 172.21.15.96:0/3757966621 >> 172.21.15.96:0/3757966621 conn(0x5556822e0550 msgr2=0x5556822e4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.779+0000 7faf15419740 1 -- 172.21.15.96:0/3757966621 shutdown_connections 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 -- 172.21.15.96:0/3757966621 wait complete. 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 -- start start 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 Processor -- start 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822eab40 0x5556822f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556822fd070 con 0x5556822e4fe0 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5556822fc9a0 con 0x5556822eab40 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5556822fd740 con 0x5556822e59d0 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf15419740 10 client.0 _ll_register_callbacks cb 0x7faf04009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36948/0 (socket says 172.21.15.96:36948) 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf0bfff640 1 -- 172.21.15.96:0/3084783022 learned_addr learned my addr 172.21.15.96:0/3084783022 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf10dd1640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822eab40 0x5556822f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.780+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 -- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 msgr2=0x5556822f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 -- 172.21.15.96:0/3084783022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822eab40 msgr2=0x5556822f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822eab40 0x5556822f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 -- 172.21.15.96:0/3084783022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556822f5d60 con 0x5556822e4fe0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf0bfff640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf0c069bd0 tx=0x7faefc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe373b303f44c46b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc01e520 con 0x5556822e4fe0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7faefc01e6c0 con 0x5556822e4fe0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf04053070 con 0x5556822e4fe0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faefc02e050 con 0x5556822e4fe0 2024-03-18T15:39:32.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555682361fb0 con 0x5556822e4fe0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf04053070 con 0x5556822e4fe0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faefc01d5b0 con 0x5556822e4fe0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 10 client.15108 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 20 client.15108 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x555682366be0 0x5556823658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 4 client.15108 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.781+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "status"}) -- 0x55568224c510 con 0x555682366be0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x555682366be0 0x5556823658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faeeb7fe640 10 client.15108 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf0b7fe640 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x555682366be0 0x5556823658f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b39c1669ca997e11 server_cookie=901c60fdadd82124 in_seq=0 out_seq=0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faeeb7fe640 1 -- 172.21.15.96:0/3084783022 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55568224c510 con 0x555682366be0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faeeb7fe640 10 client.15108 handle_command_reply: tid=0 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 client.15108 shutdown 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 msgr2=0x5556822f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faf0c069bd0 tx=0x7faefc009260 comp rx=0 tx=0).stop 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 shutdown_connections 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x555682366be0 0x5556823658f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5556822eab40 0x5556822f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5556822e59d0 0x5556822f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 --2- 172.21.15.96:0/3084783022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556822e4fe0 0x5556822f2210 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faeeb7fe640 0 client.15108 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 >> 172.21.15.96:0/3084783022 conn(0x5556822e0550 msgr2=0x5556822daaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 shutdown_connections 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 1 -- 172.21.15.96:0/3084783022 wait complete. 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.782+0000 7faf15419740 20 client.15108 trim_cache size 0 max 16384 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24472, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 84.617128502, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 109.137518443 2024-03-18T15:39:32.790 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 msgr2=0x7faef40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 0x7faef40a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf04006010 tx=0x7faf040098d0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 msgr2=0x7faf0c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e9f0 tx=0x7faf0000edf0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 shutdown_connections 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faef40a2800 0x7faef40a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf0c103450 0x7faf0c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf0c102a90 0x7faf0c198000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 --2- 172.21.15.96:0/2560984054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf0c0ffac0 0x7faf0c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 >> 172.21.15.96:0/2560984054 conn(0x7faf0c0fb3f0 msgr2=0x7faf0c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 shutdown_connections 2024-03-18T15:39:32.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.783+0000 7faf12834640 1 -- 172.21.15.96:0/2560984054 wait complete. 2024-03-18T15:39:32.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T15:39:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:32 smithi137 ceph-mon[35868]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1900840728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1019759784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2933069707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.895+0000 7f747c280640 1 -- 172.21.15.96:0/3806750134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7474102290 msgr2=0x7f7474108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.895+0000 7f747c280640 1 --2- 172.21.15.96:0/3806750134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7474102290 0x7f7474108b20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f746c009ca0 tx=0x7f746c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- 172.21.15.96:0/3806750134 shutdown_connections 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- 172.21.15.96:0/3806750134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7474102290 0x7f7474108b20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- 172.21.15.96:0/3806750134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741018d0 0x7f7474101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- 172.21.15.96:0/3806750134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741006e0 0x7f7474100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- 172.21.15.96:0/3806750134 >> 172.21.15.96:0/3806750134 conn(0x7f74740fbc70 msgr2=0x7f74740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- 172.21.15.96:0/3806750134 shutdown_connections 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- 172.21.15.96:0/3806750134 wait complete. 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 Processor -- start 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- start start 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741018d0 0x7f747419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7474102290 0x7f74741a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f747410c360 con 0x7f74741006e0 2024-03-18T15:39:32.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f747410c1e0 con 0x7f74741018d0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747c280640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f747410c4e0 con 0x7f7474102290 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747a01c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747a01c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36970/0 (socket says 172.21.15.96:36970) 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.896+0000 7f747a01c640 1 -- 172.21.15.96:0/3065255366 learned_addr learned my addr 172.21.15.96:0/3065255366 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a81d640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7474102290 0x7f74741a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747981b640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741018d0 0x7f747419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 -- 172.21.15.96:0/3065255366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7474102290 msgr2=0x7f74741a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7474102290 0x7f74741a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 -- 172.21.15.96:0/3065255366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741018d0 msgr2=0x7f747419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741018d0 0x7f747419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 -- 172.21.15.96:0/3065255366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74741a1950 con 0x7f74741006e0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747a01c640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f747000e9c0 tx=0x7f747000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f746b7fe640 1 -- 172.21.15.96:0/3065255366 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000cd80 con 0x7f74741006e0 2024-03-18T15:39:32.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f746b7fe640 1 -- 172.21.15.96:0/3065255366 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74700035f0 con 0x7f74741006e0 2024-03-18T15:39:32.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74741a1c40 con 0x7f74741006e0 2024-03-18T15:39:32.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74741a2190 con 0x7f74741006e0 2024-03-18T15:39:32.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.897+0000 7f746b7fe640 1 -- 172.21.15.96:0/3065255366 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470010690 con 0x7f74741006e0 2024-03-18T15:39:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.899+0000 7f746b7fe640 1 -- 172.21.15.96:0/3065255366 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f74700037a0 con 0x7f74741006e0 2024-03-18T15:39:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 Processor -- start 2024-03-18T15:39:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 -- start start 2024-03-18T15:39:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b4bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0aafc80 con 0x55b0c0b51730 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0a58aa0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747ee60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0aabd20 con 0x55b0c0b4bbd0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f746b7fe640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 0x7f74600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36980/0 (socket says 172.21.15.96:36980) 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 -- 172.21.15.96:0/1977022223 learned_addr learned my addr 172.21.15.96:0/1977022223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747a01c640 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747a81d640 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b4bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f746b7fe640 1 -- 172.21.15.96:0/3065255366 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f74700cee30 con 0x7f74741006e0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747a81d640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 0x7f74600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 -- 172.21.15.96:0/1977022223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4bbd0 msgr2=0x55b0c0b4bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b4bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 -- 172.21.15.96:0/1977022223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4c5c0 msgr2=0x55b0c0b510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.900+0000 7f747981b640 1 -- 172.21.15.96:0/1977022223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c0a7aff0 con 0x55b0c0b51730 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.901+0000 7f747981b640 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7464009d20 tx=0x7f7464011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f274a81bc7d8831 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.901+0000 7f747a81d640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 0x7f74600a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f746c005e30 tx=0x7f746c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.901+0000 7f7468bfe640 1 -- 172.21.15.96:0/1977022223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746401e070 con 0x55b0c0b51730 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.901+0000 7f7468bfe640 1 -- 172.21.15.96:0/1977022223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7464019b10 con 0x55b0c0b51730 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.901+0000 7f7468bfe640 1 -- 172.21.15.96:0/1977022223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400eb80 con 0x55b0c0b51730 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- 172.21.15.96:0/1977022223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 msgr2=0x55b0c0b53bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7464009d20 tx=0x7f7464011810 comp rx=0 tx=0).stop 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- 172.21.15.96:0/1977022223 shutdown_connections 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b53bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- 172.21.15.96:0/1977022223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b4bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- 172.21.15.96:0/1977022223 >> 172.21.15.96:0/1977022223 conn(0x55b0c0b47140 msgr2=0x55b0c0b47540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- 172.21.15.96:0/1977022223 shutdown_connections 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- 172.21.15.96:0/1977022223 wait complete. 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- start start 2024-03-18T15:39:32.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 Processor -- start 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b55d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b56280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b546e0 con 0x55b0c0b51730 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0a6bef0 con 0x55b0c0b4bbd0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b54860 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747ee60740 10 client.0 _ll_register_callbacks cb 0x55b0c0a66dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747a81d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747981b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b56280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747a81d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36996/0 (socket says 172.21.15.96:36996) 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747a81d640 1 -- 172.21.15.96:0/2554938508 learned_addr learned my addr 172.21.15.96:0/2554938508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.902+0000 7f747a01c640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b55d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 -- 172.21.15.96:0/2554938508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4c5c0 msgr2=0x55b0c0b56280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b56280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 -- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 msgr2=0x55b0c0b55d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b55d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 -- 172.21.15.96:0/2554938508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c0a78a30 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747a81d640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f746c009430 tx=0x7f746c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c545851419350e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c054060 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f746c004250 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7464019b10 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f746c039040 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 10 client.15120 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b0c07c42d0 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7464019b10 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f746c058020 con 0x55b0c0b51730 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 10 client.15120 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:39:32.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 20 client.15120 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.903+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0b62820 0x55b0c0b63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.904+0000 7f747ee60740 4 client.15120 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.904+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0c0abd1c0 con 0x55b0c0b62820 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.904+0000 7f747a01c640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0b62820 0x55b0c0b63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.904+0000 7f745dffb640 10 client.15120 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.904+0000 7f747a01c640 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0b62820 0x55b0c0b63c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47afc71eabb57a72 server_cookie=e12a6a9d4ad4e17b in_seq=0 out_seq=0 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f745dffb640 1 -- 172.21.15.96:0/2554938508 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b0c0abd1c0 con 0x55b0c0b62820 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f745dffb640 10 client.15120 handle_command_reply: tid=0 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 client.15120 shutdown 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 msgr2=0x55b0c0b5d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f746c009430 tx=0x7f746c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 shutdown_connections 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b51730 0x55b0c0b5d2c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b56280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0b62820 0x55b0c0b63c90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 --2- 172.21.15.96:0/2554938508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b55d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f745dffb640 0 client.15120 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:32.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 >> 172.21.15.96:0/2554938508 conn(0x55b0c0b47140 msgr2=0x55b0c0b46380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 shutdown_connections 2024-03-18T15:39:32.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 1 -- 172.21.15.96:0/2554938508 wait complete. 2024-03-18T15:39:32.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.905+0000 7f747ee60740 20 client.15120 trim_cache size 0 max 16384 2024-03-18T15:39:32.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:32 smithi193 ceph-mon[36702]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:32.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1900840728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1019759784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2933069707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:32.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 Processor -- start 2024-03-18T15:39:32.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 -- start start 2024-03-18T15:39:32.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b561e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0b50db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b61fd0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b626a0 con 0x55b0c0b4bbd0 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747ee60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b62d70 con 0x55b0c0b51730 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747981b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747981b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37006/0 (socket says 172.21.15.96:37006) 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.923+0000 7f747981b640 1 -- 172.21.15.96:0/2559280062 learned_addr learned my addr 172.21.15.96:0/2559280062 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747a01c640 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747a81d640 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0b50db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 -- 172.21.15.96:0/2559280062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 msgr2=0x55b0c0b50db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0b50db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 -- 172.21.15.96:0/2559280062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 msgr2=0x55b0c0b561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 -- 172.21.15.96:0/2559280062 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c088e6b0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747981b640 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7464009c40 tx=0x7f7464009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298b4829483084c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f745ffff640 1 -- 172.21.15.96:0/2559280062 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464028020 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f745ffff640 1 -- 172.21.15.96:0/2559280062 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74640035f0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f745ffff640 1 -- 172.21.15.96:0/2559280062 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746401e040 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747ee60740 1 -- 172.21.15.96:0/2559280062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 msgr2=0x55b0c0b5d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.924+0000 7f747ee60740 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7464009c40 tx=0x7f7464009910 comp rx=0 tx=0).stop 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- 172.21.15.96:0/2559280062 shutdown_connections 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0b50db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0b5d1e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- 172.21.15.96:0/2559280062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- 172.21.15.96:0/2559280062 >> 172.21.15.96:0/2559280062 conn(0x55b0c0b47140 msgr2=0x55b0c0b4a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- 172.21.15.96:0/2559280062 shutdown_connections 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- 172.21.15.96:0/2559280062 wait complete. 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- start start 2024-03-18T15:39:32.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 Processor -- start 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b5b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0bc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0bc6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b63b10 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b63440 con 0x55b0c0b4bbd0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0c0b641e0 con 0x55b0c0b51730 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747ee60740 10 client.0 _ll_register_callbacks cb 0x7f7464005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0bc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747a01c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b5b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747a01c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b5b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:47392/0 (socket says 172.21.15.96:47392) 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747a81d640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0bc6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747a01c640 1 -- 172.21.15.96:0/1171661406 learned_addr learned my addr 172.21.15.96:0/1171661406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 -- 172.21.15.96:0/1171661406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 msgr2=0x55b0c0bc6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0bc6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 -- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 msgr2=0x55b0c0b5b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b5b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.925+0000 7f747981b640 1 -- 172.21.15.96:0/1171661406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c0bc7410 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747981b640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0bc4570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7464011d10 tx=0x7f7464005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2158c872489d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464031070 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f7464003bb0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74640035f0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f746401e070 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 10 client.15126 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b0c0bc8710 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74640035f0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f7464019bb0 con 0x55b0c0b4c5c0 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:39:32.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 20 client.15126 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0bccbf0 0x55b0c0bcb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 4 client.15126 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "status"}) -- 0x55b0c0ab7510 con 0x55b0c0bccbf0 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.926+0000 7f747a01c640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0bccbf0 0x55b0c0bcb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f745dffb640 10 client.15126 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747a01c640 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0bccbf0 0x55b0c0bcb900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3efc36734503ec6 server_cookie=34d4cdb72ac8d563 in_seq=0 out_seq=0 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f745dffb640 1 -- 172.21.15.96:0/1171661406 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b0c0ab7510 con 0x55b0c0bccbf0 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f745dffb640 10 client.15126 handle_command_reply: tid=0 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 client.15126 shutdown 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 msgr2=0x55b0c0bc4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0bc4570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7464011d10 tx=0x7f7464005430 comp rx=0 tx=0).stop 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 shutdown_connections 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0c0b51730 0x55b0c0bc6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0c0b4c5c0 0x55b0c0bc4570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55b0c0bccbf0 0x55b0c0bcb900 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 --2- 172.21.15.96:0/1171661406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b0c0b4bbd0 0x55b0c0b5b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f745dffb640 0 client.15126 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 >> 172.21.15.96:0/1171661406 conn(0x55b0c0b47140 msgr2=0x55b0c0b4a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 shutdown_connections 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 1 -- 172.21.15.96:0/1171661406 wait complete. 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.927+0000 7f747ee60740 20 client.15126 trim_cache size 0 max 16384 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24485, 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:32.935 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 85.773216153000007, 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 94.723008241000002 2024-03-18T15:39:32.936 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 msgr2=0x7f74600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 0x7f74600a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f746c005e30 tx=0x7f746c032590 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 msgr2=0x7f7474199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f747000e9c0 tx=0x7f747000ee80 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 shutdown_connections 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f74600a2880 0x7f74600a4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7474102290 0x7f74741a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74741018d0 0x7f747419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 --2- 172.21.15.96:0/3065255366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74741006e0 0x7f7474199d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 >> 172.21.15.96:0/3065255366 conn(0x7f74740fbc70 msgr2=0x7f74740fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 shutdown_connections 2024-03-18T15:39:32.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:32.928+0000 7f747c280640 1 -- 172.21.15.96:0/3065255366 wait complete. 2024-03-18T15:39:32.939 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T15:39:33.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 -- 172.21.15.96:0/1104159546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c1006e0 msgr2=0x7fb88c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 --2- 172.21.15.96:0/1104159546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c1006e0 0x7fb88c100b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb888009be0 tx=0x7fb88802bef0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 -- 172.21.15.96:0/1104159546 shutdown_connections 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 --2- 172.21.15.96:0/1104159546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c102290 0x7fb88c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 --2- 172.21.15.96:0/1104159546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1018d0 0x7fb88c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.039+0000 7fb893aff640 1 --2- 172.21.15.96:0/1104159546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c1006e0 0x7fb88c100b00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- 172.21.15.96:0/1104159546 >> 172.21.15.96:0/1104159546 conn(0x7fb88c0fbc70 msgr2=0x7fb88c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- 172.21.15.96:0/1104159546 shutdown_connections 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- 172.21.15.96:0/1104159546 wait complete. 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 Processor -- start 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- start start 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 0x7fb88c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102290 0x7fb88c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10c330 con 0x7fb88c1006e0 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10c1b0 con 0x7fb88c1018d0 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb893aff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c10c4b0 con 0x7fb88c102290 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb89189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb89189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37030/0 (socket says 172.21.15.96:37030) 2024-03-18T15:39:33.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb89189b640 1 -- 172.21.15.96:0/1523862372 learned_addr learned my addr 172.21.15.96:0/1523862372 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb89209c640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102290 0x7fb88c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.040+0000 7fb89109a640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 0x7fb88c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 -- 172.21.15.96:0/1523862372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102290 msgr2=0x7fb88c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102290 0x7fb88c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 -- 172.21.15.96:0/1523862372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 msgr2=0x7fb88c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 0x7fb88c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 -- 172.21.15.96:0/1523862372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c19f790 con 0x7fb88c1006e0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89109a640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 0x7fb88c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb89189b640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb888036040 tx=0x7fb888033580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb882ffd640 1 -- 172.21.15.96:0/1523862372 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888043020 con 0x7fb88c1006e0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb882ffd640 1 -- 172.21.15.96:0/1523862372 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb888003e50 con 0x7fb88c1006e0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c19fa20 con 0x7fb88c1006e0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb882ffd640 1 -- 172.21.15.96:0/1523862372 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888034690 con 0x7fb88c1006e0 2024-03-18T15:39:33.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.041+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c19ff70 con 0x7fb88c1006e0 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.043+0000 7fb882ffd640 1 -- 172.21.15.96:0/1523862372 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb888046020 con 0x7fb88c1006e0 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.043+0000 7fb8967c8740 1 Processor -- start 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 -- start start 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 0x55d932cf1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cf70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932cf9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d932c55c80 con 0x55d932cf7730 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d932bfeaa0 con 0x55d932cf25c0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb8967c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d932c51d20 con 0x55d932cf1bd0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb882ffd640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 0x7fb8780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89109a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932cf9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89109a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932cf9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37032/0 (socket says 172.21.15.96:37032) 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 0x55d932cf1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89109a640 1 -- 172.21.15.96:0/1142868713 learned_addr learned my addr 172.21.15.96:0/1142868713 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cf70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 0x7fb8780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb882ffd640 1 -- 172.21.15.96:0/1523862372 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb8880f1150 con 0x7fb88c1006e0 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 -- 172.21.15.96:0/1142868713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 msgr2=0x55d932cf70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cf70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 -- 172.21.15.96:0/1142868713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 msgr2=0x55d932cf9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932cf9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.044+0000 7fb89209c640 1 -- 172.21.15.96:0/1142868713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d932c20ff0 con 0x55d932cf1bd0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb89209c640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 0x7fb8780a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb88400f790 tx=0x7fb88400e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb89209c640 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 0x55d932cf1ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8840119c0 tx=0x7fb884011e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6575b6b390865ab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb877fff640 1 -- 172.21.15.96:0/1142868713 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884011560 con 0x55d932cf1bd0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb877fff640 1 -- 172.21.15.96:0/1142868713 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb884009e00 con 0x55d932cf1bd0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb877fff640 1 -- 172.21.15.96:0/1142868713 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840035f0 con 0x55d932cf1bd0 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 -- 172.21.15.96:0/1142868713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 msgr2=0x55d932cf1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 0x55d932cf1ff0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8840119c0 tx=0x7fb884011e80 comp rx=0 tx=0).stop 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 -- 172.21.15.96:0/1142868713 shutdown_connections 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932cf9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cf70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1142868713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf1bd0 0x55d932cf1ff0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.045+0000 7fb8967c8740 1 -- 172.21.15.96:0/1142868713 >> 172.21.15.96:0/1142868713 conn(0x55d932ced140 msgr2=0x55d932ced540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- 172.21.15.96:0/1142868713 shutdown_connections 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- 172.21.15.96:0/1142868713 wait complete. 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- start start 2024-03-18T15:39:33.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 Processor -- start 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932cfbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf25c0 0x55d932cfc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d932cfa680 con 0x55d932cf7730 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d932c11ef0 con 0x55d932cf1bd0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d932cfa800 con 0x55d932cf25c0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb8967c8740 10 client.0 _ll_register_callbacks cb 0x55d932c0cdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb89209c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb89209c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37036/0 (socket says 172.21.15.96:37036) 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb89189b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932cfbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb89209c640 1 -- 172.21.15.96:0/1292306953 learned_addr learned my addr 172.21.15.96:0/1292306953 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.046+0000 7fb89109a640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf25c0 0x55d932cfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 -- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf25c0 msgr2=0x55d932cfc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf25c0 0x55d932cfc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 -- 172.21.15.96:0/1292306953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 msgr2=0x55d932cfbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932cfbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 -- 172.21.15.96:0/1292306953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d932c1ea30 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89189b640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932cfbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb89209c640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb884011990 tx=0x7fb8840048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c808e910e1da6f97 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884003a00 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb884003ba0 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb884009e00 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb884003da0 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8967c8740 10 client.15138 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d93296a2d0 con 0x55d932cf7730 2024-03-18T15:39:33.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb884009e00 con 0x55d932cf7730 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb884026610 con 0x55d932cf7730 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8967c8740 10 client.15138 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.047+0000 7fb8967c8740 10 client.15138 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb8967c8740 20 client.15138 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d086e0 0x55d932d09b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb8967c8740 4 client.15138 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d932c631c0 con 0x55d932d086e0 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb89189b640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d086e0 0x55d932d09b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb8757fa640 10 client.15138 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.048+0000 7fb89189b640 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d086e0 0x55d932d09b50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb5a8dbb5eb72b0 server_cookie=50d1f51ecd211176 in_seq=0 out_seq=0 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8757fa640 1 -- 172.21.15.96:0/1292306953 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d932c631c0 con 0x55d932d086e0 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8757fa640 10 client.15138 handle_command_reply: tid=0 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 client.15138 shutdown 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 msgr2=0x55d932d03340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb884011990 tx=0x7fb8840048d0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 shutdown_connections 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d086e0 0x55d932d09b50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf7730 0x55d932d03340 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8757fa640 0 client.15138 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf25c0 0x55d932cfc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 --2- 172.21.15.96:0/1292306953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932cfbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 >> 172.21.15.96:0/1292306953 conn(0x55d932ced140 msgr2=0x55d932cec380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 shutdown_connections 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 1 -- 172.21.15.96:0/1292306953 wait complete. 2024-03-18T15:39:33.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.049+0000 7fb8967c8740 20 client.15138 trim_cache size 0 max 16384 2024-03-18T15:39:33.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 Processor -- start 2024-03-18T15:39:33.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 -- start start 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932d07680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932cf3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d932d07e90 con 0x55d932cf25c0 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d932d08560 con 0x55d932cf1bd0 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb8967c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d932d08c30 con 0x55d932cf7730 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37052/0 (socket says 172.21.15.96:37052) 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 -- 172.21.15.96:0/2234227901 learned_addr learned my addr 172.21.15.96:0/2234227901 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89209c640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932cf3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89189b640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932d07680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 -- 172.21.15.96:0/2234227901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 msgr2=0x55d932cf3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932cf3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 -- 172.21.15.96:0/2234227901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 msgr2=0x55d932d07680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932d07680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 -- 172.21.15.96:0/2234227901 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d932a346b0 con 0x55d932cf25c0 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89209c640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932cf3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.067+0000 7fb89109a640 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb87c004620 tx=0x7fb87c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae1ceb2b20e77b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8777fe640 1 -- 172.21.15.96:0/2234227901 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c003ad0 con 0x55d932cf25c0 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8777fe640 1 -- 172.21.15.96:0/2234227901 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c003c70 con 0x55d932cf25c0 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8777fe640 1 -- 172.21.15.96:0/2234227901 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0116a0 con 0x55d932cf25c0 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- 172.21.15.96:0/2234227901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 msgr2=0x55d932d03300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb87c004620 tx=0x7fb87c009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- 172.21.15.96:0/2234227901 shutdown_connections 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932cf3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf25c0 0x55d932d03300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 --2- 172.21.15.96:0/2234227901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf1bd0 0x55d932d07680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- 172.21.15.96:0/2234227901 >> 172.21.15.96:0/2234227901 conn(0x55d932ced140 msgr2=0x55d932cf6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- 172.21.15.96:0/2234227901 shutdown_connections 2024-03-18T15:39:33.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- 172.21.15.96:0/2234227901 wait complete. 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.068+0000 7fb8967c8740 1 -- start start 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 Processor -- start 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cfed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932d01d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d932d099d0 con 0x55d932cf1bd0 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d932d09300 con 0x55d932cf25c0 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d932d0a0a0 con 0x55d932cf7730 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8967c8740 10 client.0 _ll_register_callbacks cb 0x7fb87c0095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37068/0 (socket says 172.21.15.96:37068) 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 -- 172.21.15.96:0/3705315186 learned_addr learned my addr 172.21.15.96:0/3705315186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89109a640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cfed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89209c640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932d01d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 -- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 msgr2=0x55d932d01d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932d01d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 -- 172.21.15.96:0/3705315186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 msgr2=0x55d932cfed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cfed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 -- 172.21.15.96:0/3705315186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d932d02300 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb89189b640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8880099a0 tx=0x7fb888034eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab06048d850dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.069+0000 7fb8757fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880bdae0 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb88804d200 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87c003c70 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb8880c1c30 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 10 client.15144 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d932d03600 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87c003c70 con 0x55d932cf1bd0 2024-03-18T15:39:33.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb8880d6ba0 con 0x55d932cf1bd0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 10 client.15144 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 10 client.15144 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 20 client.15144 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d72ec0 0x55d932d71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 4 client.15144 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "status"}) -- 0x55d932c5d510 con 0x55d932d72ec0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.070+0000 7fb89109a640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d72ec0 0x55d932d71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8757fa640 10 client.15144 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb89109a640 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d72ec0 0x55d932d71bd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12b04e899f61dcf server_cookie=ea53661a695ed5a8 in_seq=0 out_seq=0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8757fa640 1 -- 172.21.15.96:0/3705315186 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d932c5d510 con 0x55d932d72ec0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8757fa640 10 client.15144 handle_command_reply: tid=0 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 client.15144 shutdown 2024-03-18T15:39:33.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 msgr2=0x55d932cfe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8880099a0 tx=0x7fb888034eb0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 shutdown_connections 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d932cf7730 0x55d932d01d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d932cf25c0 0x55d932cfed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d932d72ec0 0x55d932d71bd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 --2- 172.21.15.96:0/3705315186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d932cf1bd0 0x55d932cfe800 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8757fa640 0 client.15144 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 >> 172.21.15.96:0/3705315186 conn(0x55d932ced140 msgr2=0x55d932d07680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 shutdown_connections 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 1 -- 172.21.15.96:0/3705315186 wait complete. 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.071+0000 7fb8967c8740 20 client.15144 trim_cache size 0 max 16384 2024-03-18T15:39:33.079 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24493, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 88.257725531999995, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 103.638652781 2024-03-18T15:39:33.080 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 msgr2=0x7fb8780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 0x7fb8780a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb88400f790 tx=0x7fb88400e000 comp rx=0 tx=0).stop 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 msgr2=0x7fb88c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb888036040 tx=0x7fb888033580 comp rx=0 tx=0).stop 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 shutdown_connections 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb8780a2800 0x7fb8780a4cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c102290 0x7fb88c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb88c1018d0 0x7fb88c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 --2- 172.21.15.96:0/1523862372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb88c1006e0 0x7fb88c197bc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:33.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 >> 172.21.15.96:0/1523862372 conn(0x7fb88c0fbc70 msgr2=0x7fb88c0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:33.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 shutdown_connections 2024-03-18T15:39:33.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:33.072+0000 7fb893aff640 1 -- 172.21.15.96:0/1523862372 wait complete. 2024-03-18T15:39:33.083 INFO:tasks.cephadm:Setup complete, yielding 2024-03-18T15:39:33.083 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T15:39:33.099 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:39:33.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-18T15:39:33.336 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:33.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:33 smithi096 ceph-mon[31930]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T15:39:33.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:33 smithi137 ceph-mon[35868]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T15:39:33.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:33 smithi193 ceph-mon[36702]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-18T15:39:34.822 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:34 smithi096 ceph-mon[31930]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:34 smithi137 ceph-mon[35868]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:34.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:34 smithi193 ceph-mon[36702]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- 172.21.15.96:0/2456025607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600108050 msgr2=0x7f960010a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 --2- 172.21.15.96:0/2456025607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600108050 0x7f960010a460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f95fc009990 tx=0x7f95fc009ca0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- 172.21.15.96:0/2456025607 shutdown_connections 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 --2- 172.21.15.96:0/2456025607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600108050 0x7f960010a460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 --2- 172.21.15.96:0/2456025607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600102df0 0x7f96001051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 --2- 172.21.15.96:0/2456025607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600105720 0x7f9600107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- 172.21.15.96:0/2456025607 >> 172.21.15.96:0/2456025607 conn(0x7f96000fc7c0 msgr2=0x7f96000febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- 172.21.15.96:0/2456025607 shutdown_connections 2024-03-18T15:39:35.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- 172.21.15.96:0/2456025607 wait complete. 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 Processor -- start 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.037+0000 7f9607d9a700 1 -- start start 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600105720 0x7f9600100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600108050 0x7f96001012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f960010ded0 con 0x7f9600102df0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f960010dd50 con 0x7f9600108050 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f960010e050 con 0x7f9600105720 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45598/0 (socket says 172.21.15.96:45598) 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 -- 172.21.15.96:0/3832890042 learned_addr learned my addr 172.21.15.96:0/3832890042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605335700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600105720 0x7f9600100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9606337700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600108050 0x7f96001012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 -- 172.21.15.96:0/3832890042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600105720 msgr2=0x7f9600100da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600105720 0x7f9600100da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 -- 172.21.15.96:0/3832890042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600108050 msgr2=0x7f96001012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600108050 0x7f96001012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 -- 172.21.15.96:0/3832890042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9600101b90 con 0x7f9600102df0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9605b36700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95f800ef10 tx=0x7f95f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f800c9c0 con 0x7f9600102df0 2024-03-18T15:39:35.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f8019070 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96001a25d0 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007e60 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.038+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a2ae0 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.039+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9600068f90 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.040+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f95f8012050 con 0x7f9600102df0 2024-03-18T15:39:35.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.041+0000 7f95f6ffd700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 0x7f95ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:35.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.041+0000 7f9605335700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 0x7f95ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:35.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.041+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f95f80ce9d0 con 0x7f9600102df0 2024-03-18T15:39:35.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.041+0000 7f9605335700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 0x7f95ec0a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009c60 tx=0x7f95f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:35.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.045+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f95f8010040 con 0x7f9600102df0 2024-03-18T15:39:35.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.260+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9600062710 con 0x7f95ec0a2890 2024-03-18T15:39:35.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.261+0000 7f95f6ffd700 1 -- 172.21.15.96:0/3832890042 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9600062710 con 0x7f95ec0a2890 2024-03-18T15:39:35.262 INFO:teuthology.orchestra.run.smithi096.stdout:Backend: cephadm 2024-03-18T15:39:35.262 INFO:teuthology.orchestra.run.smithi096.stdout:Available: Yes 2024-03-18T15:39:35.262 INFO:teuthology.orchestra.run.smithi096.stdout:Paused: No 2024-03-18T15:39:35.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 msgr2=0x7f95ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:35.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 0x7f95ec0a4d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009c60 tx=0x7f95f0009260 comp rx=0 tx=0).stop 2024-03-18T15:39:35.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 msgr2=0x7f9600102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95f800ef10 tx=0x7f95f800c3a0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 shutdown_connections 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95ec0a2890 0x7f95ec0a4d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9600108050 0x7f96001012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9600102df0 0x7f9600102750 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 --2- 172.21.15.96:0/3832890042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9600105720 0x7f9600100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.263+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 >> 172.21.15.96:0/3832890042 conn(0x7f96000fc7c0 msgr2=0x7f9600103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.264+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 shutdown_connections 2024-03-18T15:39:35.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:35.264+0000 7f9607d9a700 1 -- 172.21.15.96:0/3832890042 wait complete. 2024-03-18T15:39:36.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-18T15:39:36.253 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:36.637 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:36 smithi096 ceph-mon[31930]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:36.637 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:36 smithi096 ceph-mon[31930]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:36 smithi137 ceph-mon[35868]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:36 smithi137 ceph-mon[35868]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:36 smithi193 ceph-mon[36702]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:36 smithi193 ceph-mon[36702]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 -- 172.21.15.96:0/1264927446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 msgr2=0x7f06ac102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1264927446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac102850 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f069c0099c0 tx=0x7f069c009cd0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 -- 172.21.15.96:0/1264927446 shutdown_connections 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1264927446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 0x7f06ac109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1264927446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 0x7f06ac101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1264927446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac102850 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 -- 172.21.15.96:0/1264927446 >> 172.21.15.96:0/1264927446 conn(0x7f06ac0fc790 msgr2=0x7f06ac0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:38.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 -- 172.21.15.96:0/1264927446 shutdown_connections 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.003+0000 7f06b37e0700 1 -- 172.21.15.96:0/1264927446 wait complete. 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 Processor -- start 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 -- start start 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 0x7f06ac1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 0x7f06ac19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac10cdf0 con 0x7f06ac1023f0 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac10cc70 con 0x7f06ac1011f0 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b37e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06ac10cf70 con 0x7f06ac102d90 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b0d7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b0d7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45618/0 (socket says 172.21.15.96:45618) 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.004+0000 7f06b0d7b700 1 -- 172.21.15.96:0/1016853992 learned_addr learned my addr 172.21.15.96:0/1016853992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:38.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b157c700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 0x7f06ac1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b1d7d700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 0x7f06ac19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 -- 172.21.15.96:0/1016853992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 msgr2=0x7f06ac19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 0x7f06ac19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 -- 172.21.15.96:0/1016853992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 msgr2=0x7f06ac1973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 0x7f06ac1973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 -- 172.21.15.96:0/1016853992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ac06a910 con 0x7f06ac1023f0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b0d7b700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f069c003790 tx=0x7f069c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c009e60 con 0x7f06ac1023f0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ac06aba0 con 0x7f06ac1023f0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f069c003960 con 0x7f06ac1023f0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0173f0 con 0x7f06ac1023f0 2024-03-18T15:39:38.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.005+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ac06b080 con 0x7f06ac1023f0 2024-03-18T15:39:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.006+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ac068f90 con 0x7f06ac1023f0 2024-03-18T15:39:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.007+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f069c003b10 con 0x7f06ac1023f0 2024-03-18T15:39:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.008+0000 7f06a27fc700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 0x7f06980a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.008+0000 7f06b157c700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 0x7f06980a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:38.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.008+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f069c0cf810 con 0x7f06ac1023f0 2024-03-18T15:39:38.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.008+0000 7f06b157c700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 0x7f06980a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f06ac102250 tx=0x7f06a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:38.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.012+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f069c013070 con 0x7f06ac1023f0 2024-03-18T15:39:38.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.218+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f06ac06b2e0 con 0x7f06980a2890 2024-03-18T15:39:38.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.231+0000 7f06a27fc700 1 -- 172.21.15.96:0/1016853992 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f06ac06b2e0 con 0x7f06980a2890 2024-03-18T15:39:38.232 INFO:teuthology.orchestra.run.smithi096.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-18T15:39:38.232 INFO:teuthology.orchestra.run.smithi096.stdout:mds.a smithi096 running (103s) 16s ago 103s 18.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 91c4375e4ee8 2024-03-18T15:39:38.232 INFO:teuthology.orchestra.run.smithi096.stdout:mds.b smithi137 running (100s) 33s ago 99s 18.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 669b3735fb36 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.c smithi193 running (114s) 33s ago 114s 19.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b f80cfa8af247 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.d smithi096 running (2m) 16s ago 2m 16.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 3ed1546d7ec7 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.e smithi137 running (2m) 33s ago 119s 17.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b dda81e459d0d 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.f smithi193 running (117s) 33s ago 117s 19.2M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 4fb09f2ce5ec 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.g smithi096 running (106s) 16s ago 105s 16.4M - 19.0.0-2162-g3d78f5ba be86dc8cb22b cffe54d2f7fe 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.h smithi137 running (112s) 33s ago 111s 17.5M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 860d950f7d70 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.i smithi193 running (109s) 33s ago 108s 28.7M - 19.0.0-2162-g3d78f5ba be86dc8cb22b d469ff70a54d 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.j smithi096 running (2m) 16s ago 2m 13.9M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 3a3f33e59064 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.k smithi137 running (2m) 33s ago 2m 19.0M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 8f3ca3132884 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mds.l smithi193 running (2m) 33s ago 2m 19.1M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 05d3bb9f0d0c 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.x smithi096 *:9283,8765 running (9m) 16s ago 9m 513M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 258b43739b09 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.y smithi137 *:8443,8765 running (7m) 33s ago 7m 454M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 55f7d96e1a07 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.z smithi193 *:8443,8765 running (7m) 33s ago 7m 452M - 19.0.0-2162-g3d78f5ba be86dc8cb22b 9405ffcae5ef 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a smithi096 running (9m) 16s ago 9m 50.7M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 6182999c5ae5 2024-03-18T15:39:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b smithi137 running (7m) 33s ago 7m 43.5M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 3b5324c405f8 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c smithi193 running (7m) 33s ago 7m 41.1M 2048M 19.0.0-2162-g3d78f5ba be86dc8cb22b 8db2e4fe0093 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 smithi096 running (7m) 16s ago 7m 45.6M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 52eb444efd52 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 smithi137 running (6m) 33s ago 6m 58.1M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b b42c20c9ce2f 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 smithi193 running (6m) 33s ago 6m 56.9M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5d4fe8cd76af 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 smithi096 running (5m) 16s ago 5m 52.5M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 72d239c58066 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 smithi137 running (5m) 33s ago 5m 41.6M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 77252dac3187 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 smithi193 running (5m) 33s ago 5m 58.1M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b bc288e6b0f11 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 smithi096 running (4m) 16s ago 4m 55.3M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 2a6a360f56fb 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 smithi137 running (4m) 33s ago 4m 54.6M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 6618498ccffe 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 smithi193 running (3m) 33s ago 3m 42.7M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 7b866f576694 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 smithi096 running (3m) 16s ago 3m 41.5M 4272M 19.0.0-2162-g3d78f5ba be86dc8cb22b 5c266ad593c8 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 smithi137 running (3m) 33s ago 2m 41.8M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 8b8a5554cb5b 2024-03-18T15:39:38.234 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 smithi193 running (2m) 33s ago 2m 53.2M 4270M 19.0.0-2162-g3d78f5ba be86dc8cb22b 691eb1b9c7e8 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 msgr2=0x7f06980a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 0x7f06980a4d50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f06ac102250 tx=0x7f06a8009260 comp rx=0 tx=0).stop 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 msgr2=0x7f06ac197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f069c003790 tx=0x7f069c00f690 comp rx=0 tx=0).stop 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 shutdown_connections 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f06980a2890 0x7f06980a4d50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ac102d90 0x7f06ac19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06ac1011f0 0x7f06ac1973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 --2- 172.21.15.96:0/1016853992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06ac1023f0 0x7f06ac197910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:38.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 >> 172.21.15.96:0/1016853992 conn(0x7f06ac0fc790 msgr2=0x7f06ac105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:38.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 shutdown_connections 2024-03-18T15:39:38.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:38.234+0000 7f06b37e0700 1 -- 172.21.15.96:0/1016853992 wait complete. 2024-03-18T15:39:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:38 smithi137 ceph-mon[35868]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:39:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:38 smithi193 ceph-mon[36702]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:39:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:38 smithi096 ceph-mon[31930]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:39:39.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-18T15:39:39.276 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:39.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:39 smithi096 ceph-mon[31930]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:39.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:39 smithi137 ceph-mon[35868]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:39.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:39 smithi193 ceph-mon[36702]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:40.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:40 smithi096 ceph-mon[31930]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:40 smithi137 ceph-mon[35868]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:40.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:40 smithi193 ceph-mon[36702]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:41.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 -- 172.21.15.96:0/91903516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103ea0 msgr2=0x7f14a010a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 --2- 172.21.15.96:0/91903516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103ea0 0x7f14a010a750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f149800b0d0 tx=0x7f149800b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 -- 172.21.15.96:0/91903516 shutdown_connections 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 --2- 172.21.15.96:0/91903516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103ea0 0x7f14a010a750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 --2- 172.21.15.96:0/91903516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 --2- 172.21.15.96:0/91903516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103500 0x7f14a0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 -- 172.21.15.96:0/91903516 >> 172.21.15.96:0/91903516 conn(0x7f14a00fd8e0 msgr2=0x7f14a00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 -- 172.21.15.96:0/91903516 shutdown_connections 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.049+0000 7f14a554a700 1 -- 172.21.15.96:0/91903516 wait complete. 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 Processor -- start 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 -- start start 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:41.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103ea0 0x7f14a0074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010dee0 con 0x7f14a0103500 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010dd60 con 0x7f14a0103ea0 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f14a554a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14a010e060 con 0x7f14a0102300 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45634/0 (socket says 172.21.15.96:45634) 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 -- 172.21.15.96:0/85920359 learned_addr learned my addr 172.21.15.96:0/85920359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149f7fe700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103ea0 0x7f14a0074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 -- 172.21.15.96:0/85920359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 msgr2=0x7f14a0076070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0076070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 -- 172.21.15.96:0/85920359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103ea0 msgr2=0x7f14a0074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103ea0 0x7f14a0074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149e7fc700 1 -- 172.21.15.96:0/85920359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a00754b0 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.050+0000 7f149effd700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f149e7fc700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f149400d980 tx=0x7f149400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494016070 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a01a2630 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1494004d10 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494005670 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.051+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a01a2b40 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.052+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a0068f90 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.053+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f1494004750 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.053+0000 7f1487fff700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 0x7f14880a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.053+0000 7f149effd700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 0x7f14880a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.053+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f14940ce810 con 0x7f14a0103500 2024-03-18T15:39:41.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.054+0000 7f149effd700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 0x7f14880a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14a0103360 tx=0x7f1490009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:41.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.058+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f149401f020 con 0x7f14a0103500 2024-03-18T15:39:41.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.266+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f14a0062710 con 0x7f14880a2890 2024-03-18T15:39:41.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.273+0000 7f1487fff700 1 -- 172.21.15.96:0/85920359 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f14a0062710 con 0x7f14880a2890 2024-03-18T15:39:41.274 INFO:teuthology.orchestra.run.smithi096.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-18T15:39:41.274 INFO:teuthology.orchestra.run.smithi096.stdout:mds.all 12/12 36s ago 2m smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-03-18T15:39:41.274 INFO:teuthology.orchestra.run.smithi096.stdout:mgr 3/3 36s ago 7m smithi096=x;smithi137=y;smithi193=z;count:3 2024-03-18T15:39:41.274 INFO:teuthology.orchestra.run.smithi096.stdout:mon 3/3 36s ago 8m smithi096:172.21.15.96=a;smithi137:172.21.15.137=b;smithi193:172.21.15.193=c;count:3 2024-03-18T15:39:41.274 INFO:teuthology.orchestra.run.smithi096.stdout:osd 12 36s ago - 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 msgr2=0x7f14880a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 0x7f14880a4d50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14a0103360 tx=0x7f1490009260 comp rx=0 tx=0).stop 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 msgr2=0x7f14a00746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f149400d980 tx=0x7f149400dc90 comp rx=0 tx=0).stop 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 shutdown_connections 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14880a2890 0x7f14880a4d50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14a0103ea0 0x7f14a0074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14a0102300 0x7f14a0076070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 --2- 172.21.15.96:0/85920359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14a0103500 0x7f14a00746c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:41.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 >> 172.21.15.96:0/85920359 conn(0x7f14a00fd8e0 msgr2=0x7f14a0106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:41.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 shutdown_connections 2024-03-18T15:39:41.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:41.275+0000 7f14a554a700 1 -- 172.21.15.96:0/85920359 wait complete. 2024-03-18T15:39:41.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-18T15:39:42.222 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:42.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:42 smithi096 ceph-mon[31930]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:42.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:42 smithi096 ceph-mon[31930]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:42 smithi137 ceph-mon[35868]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:42 smithi137 ceph-mon[35868]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:42.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:42 smithi193 ceph-mon[36702]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:42 smithi193 ceph-mon[36702]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:43.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- 172.21.15.96:0/1679955177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 msgr2=0x7f47d810a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 --2- 172.21.15.96:0/1679955177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d810a750 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f47d000b0d0 tx=0x7f47d000b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- 172.21.15.96:0/1679955177 shutdown_connections 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 --2- 172.21.15.96:0/1679955177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d810a750 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 --2- 172.21.15.96:0/1679955177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8102300 0x7f47d8102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 --2- 172.21.15.96:0/1679955177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8103500 0x7f47d8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- 172.21.15.96:0/1679955177 >> 172.21.15.96:0/1679955177 conn(0x7f47d80fd8e0 msgr2=0x7f47d80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- 172.21.15.96:0/1679955177 shutdown_connections 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- 172.21.15.96:0/1679955177 wait complete. 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 Processor -- start 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.945+0000 7f47dd907700 1 -- start start 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 0x7f47d8076070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8103500 0x7f47d80746c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47d810df40 con 0x7f47d8103ea0 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47d810ddc0 con 0x7f47d8102300 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dd907700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f47d810e0c0 con 0x7f47d8103500 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d77fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41694/0 (socket says 172.21.15.96:41694) 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 0x7f47d8076070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d77fe700 1 -- 172.21.15.96:0/2698672149 learned_addr learned my addr 172.21.15.96:0/2698672149 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47ce7fc700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8103500 0x7f47d80746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 -- 172.21.15.96:0/2698672149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8103500 msgr2=0x7f47d80746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8103500 0x7f47d80746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 -- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 msgr2=0x7f47d8074c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 -- 172.21.15.96:0/2698672149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d80754b0 con 0x7f47d8102300 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d77fe700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:43.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47d6ffd700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 0x7f47d8076070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9c0 tx=0x7f47c800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.946+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8004230 con 0x7f47d8102300 2024-03-18T15:39:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.947+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d81a2650 con 0x7f47d8102300 2024-03-18T15:39:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.947+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c80043d0 con 0x7f47d8102300 2024-03-18T15:39:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.947+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8010430 con 0x7f47d8102300 2024-03-18T15:39:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.947+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d81a2b60 con 0x7f47d8102300 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.947+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d8068f90 con 0x7f47d8102300 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.949+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f47c800c7c0 con 0x7f47d8102300 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.949+0000 7f47dc905700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 0x7f47c40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.949+0000 7f47ce7fc700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 0x7f47c40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.949+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f47c80ced80 con 0x7f47d8102300 2024-03-18T15:39:43.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.950+0000 7f47ce7fc700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 0x7f47c40a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009c60 tx=0x7f47c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:43.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:43.957+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f47c809be10 con 0x7f47d8102300 2024-03-18T15:39:44.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.159+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f47d8062710 con 0x7f47c40a2890 2024-03-18T15:39:44.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.160+0000 7f47dc905700 1 -- 172.21.15.96:0/2698672149 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f47d8062710 con 0x7f47c40a2890 2024-03-18T15:39:44.161 INFO:teuthology.orchestra.run.smithi096.stdout:HOST ADDR LABELS STATUS 2024-03-18T15:39:44.161 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 172.21.15.96 2024-03-18T15:39:44.162 INFO:teuthology.orchestra.run.smithi096.stdout:smithi137 172.21.15.137 2024-03-18T15:39:44.162 INFO:teuthology.orchestra.run.smithi096.stdout:smithi193 172.21.15.193 2024-03-18T15:39:44.162 INFO:teuthology.orchestra.run.smithi096.stdout:3 hosts in cluster 2024-03-18T15:39:44.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 msgr2=0x7f47c40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:44.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 0x7f47c40a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009c60 tx=0x7f47c0009260 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 msgr2=0x7f47d8076070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 0x7f47d8076070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9c0 tx=0x7f47c800ecd0 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 shutdown_connections 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f47c40a2890 0x7f47c40a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47d8103ea0 0x7f47d8074c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47d8102300 0x7f47d8076070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.162+0000 7f47dd907700 1 --2- 172.21.15.96:0/2698672149 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47d8103500 0x7f47d80746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.163+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 >> 172.21.15.96:0/2698672149 conn(0x7f47d80fd8e0 msgr2=0x7f47d81067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.163+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 shutdown_connections 2024-03-18T15:39:44.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:44.163+0000 7f47dd907700 1 -- 172.21.15.96:0/2698672149 wait complete. 2024-03-18T15:39:44.804 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:44 smithi096 ceph-mon[31930]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:44.833 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-18T15:39:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:44 smithi137 ceph-mon[35868]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:44 smithi193 ceph-mon[36702]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-18T15:39:45.072 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:45.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:45 smithi137 ceph-mon[35868]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:45.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:45 smithi193 ceph-mon[36702]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:45 smithi096 ceph-mon[31930]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:46.759 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:46 smithi096 ceph-mon[31930]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:46 smithi137 ceph-mon[35868]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 -- 172.21.15.96:0/1655288182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108080 msgr2=0x7fa14c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 --2- 172.21.15.96:0/1655288182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108080 0x7fa14c10a490 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b0a0 tx=0x7fa14400b3b0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 -- 172.21.15.96:0/1655288182 shutdown_connections 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 --2- 172.21.15.96:0/1655288182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c108080 0x7fa14c10a490 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 --2- 172.21.15.96:0/1655288182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c102e20 0x7fa14c105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 --2- 172.21.15.96:0/1655288182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c105750 0x7fa14c107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 -- 172.21.15.96:0/1655288182 >> 172.21.15.96:0/1655288182 conn(0x7fa14c0fc7f0 msgr2=0x7fa14c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 -- 172.21.15.96:0/1655288182 shutdown_connections 2024-03-18T15:39:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.861+0000 7fa151e8e700 1 -- 172.21.15.96:0/1655288182 wait complete. 2024-03-18T15:39:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 Processor -- start 2024-03-18T15:39:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 -- start start 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c105750 0x7fa14c100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c108080 0x7fa14c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10df40 con 0x7fa14c102e20 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10ddc0 con 0x7fa14c108080 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa151e8e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10e0c0 con 0x7fa14c105750 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa14b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa14b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41714/0 (socket says 172.21.15.96:41714) 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa14b7fe700 1 -- 172.21.15.96:0/3333128413 learned_addr learned my addr 172.21.15.96:0/3333128413 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa14affd700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c105750 0x7fa14c100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.862+0000 7fa14bfff700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c108080 0x7fa14c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 -- 172.21.15.96:0/3333128413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c105750 msgr2=0x7fa14c100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c105750 0x7fa14c100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 -- 172.21.15.96:0/3333128413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c108080 msgr2=0x7fa14c101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c108080 0x7fa14c101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 -- 172.21.15.96:0/3333128413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c101b50 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa14b7fe700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9c0 tx=0x7fa13c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00cbd0 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c00ee60 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c1a25f0 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c005630 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.863+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a2b30 con 0x7fa14c102e20 2024-03-18T15:39:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.864+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c068f90 con 0x7fa14c102e20 2024-03-18T15:39:46.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.865+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fa13c004750 con 0x7fa14c102e20 2024-03-18T15:39:46.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.865+0000 7fa148ff9700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.866+0000 7fa14affd700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.866+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fa13c0cec50 con 0x7fa14c102e20 2024-03-18T15:39:46.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.866+0000 7fa14affd700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 0x7fa1340a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa140009c60 tx=0x7fa140009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:46.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:46.870+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa13c0d4020 con 0x7fa14c102e20 2024-03-18T15:39:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:46 smithi193 ceph-mon[36702]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:39:47.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.075+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa14c062710 con 0x7fa1340a2910 2024-03-18T15:39:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.077+0000 7fa148ff9700 1 -- 172.21.15.96:0/3333128413 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa14c062710 con 0x7fa1340a2910 2024-03-18T15:39:47.077 INFO:teuthology.orchestra.run.smithi096.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-18T15:39:47.078 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-18T15:39:47.078 INFO:teuthology.orchestra.run.smithi096.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-18T15:39:47.078 INFO:teuthology.orchestra.run.smithi096.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-18T15:39:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 msgr2=0x7fa1340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 0x7fa1340a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa140009c60 tx=0x7fa140009260 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 msgr2=0x7fa14c102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00e9c0 tx=0x7fa13c00ecd0 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 shutdown_connections 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa14c108080 0x7fa14c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa14c102e20 0x7fa14c102710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 --2- 172.21.15.96:0/3333128413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa14c105750 0x7fa14c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 >> 172.21.15.96:0/3333128413 conn(0x7fa14c0fc7f0 msgr2=0x7fa14c1037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.079+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 shutdown_connections 2024-03-18T15:39:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:47.080+0000 7fa151e8e700 1 -- 172.21.15.96:0/3333128413 wait complete. 2024-03-18T15:39:47.792 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-18T15:39:47.802 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:39:47.802 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-18T15:39:48.042 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:48 smithi096 ceph-mon[31930]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:48.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:48 smithi096 ceph-mon[31930]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:48 smithi137 ceph-mon[35868]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:48.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:48 smithi137 ceph-mon[35868]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:48.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:48 smithi193 ceph-mon[36702]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:39:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:48 smithi193 ceph-mon[36702]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 -- 172.21.15.96:0/1286298279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294102da0 msgr2=0x7f2294109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 --2- 172.21.15.96:0/1286298279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294102da0 0x7f2294109650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f228c00b0d0 tx=0x7f228c00b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 -- 172.21.15.96:0/1286298279 shutdown_connections 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 --2- 172.21.15.96:0/1286298279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294102da0 0x7f2294109650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 --2- 172.21.15.96:0/1286298279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294101200 0x7f2294101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 --2- 172.21.15.96:0/1286298279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102400 0x7f2294102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 -- 172.21.15.96:0/1286298279 >> 172.21.15.96:0/1286298279 conn(0x7f22940fc7a0 msgr2=0x7f22940febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 -- 172.21.15.96:0/1286298279 shutdown_connections 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.761+0000 7f229b479700 1 -- 172.21.15.96:0/1286298279 wait complete. 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 Processor -- start 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 -- start start 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294102400 0x7f22941037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 0x7f2294103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f229410ce00 con 0x7f2294101200 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f229410cc80 con 0x7f2294102400 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f229b479700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f229410cf80 con 0x7f2294102da0 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299215700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299215700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41722/0 (socket says 172.21.15.96:41722) 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299215700 1 -- 172.21.15.96:0/565186160 learned_addr learned my addr 172.21.15.96:0/565186160 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:49.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 0x7f2294103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2298a14700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294102400 0x7f22941037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 -- 172.21.15.96:0/565186160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294102400 msgr2=0x7f22941037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294102400 0x7f22941037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 -- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 msgr2=0x7f2294105190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299a16700 1 -- 172.21.15.96:0/565186160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22941045d0 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.762+0000 7f2299215700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f2299a16700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 0x7f2294103d20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f228c00b0d0 tx=0x7f228c007a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c021070 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22941a2640 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f228c01db30 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228c01bb50 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.763+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22941a2aa0 con 0x7f2294102da0 2024-03-18T15:39:49.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.764+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2294068f90 con 0x7f2294102da0 2024-03-18T15:39:49.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.765+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f228c024070 con 0x7f2294102da0 2024-03-18T15:39:49.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.765+0000 7f228a7fc700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 0x7f22800a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:49.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.765+0000 7f2299215700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 0x7f22800a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:49.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.766+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f228c0cfd20 con 0x7f2294102da0 2024-03-18T15:39:49.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.766+0000 7f2299215700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 0x7f22800a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2284007fd0 tx=0x7f2284007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:49.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:49.769+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f228c09ce40 con 0x7f2294102da0 2024-03-18T15:39:50.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.009+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f22941a2e90 con 0x7f2294102da0 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.010+0000 7f228a7fc700 1 -- 172.21.15.96:0/565186160 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3092 (secure 0 0 0) 0x7f228c0bc070 con 0x7f2294102da0 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:e24 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:legacy client fscid: 1 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem 'cephfs' (1) 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:fs_name cephfs 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 24 2024-03-18T15:39:50.011 INFO:teuthology.orchestra.run.smithi096.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-03-18T15:38:04.167866+0000 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-03-18T15:38:11.142722+0000 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:tableserver 0 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:root 0 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:session_timeout 300 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:session_autoclose 300 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:max_file_size 1099511627776 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:max_xattr_size 65536 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:required_client_features {} 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure 0 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure_osd_epoch 0 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:max_mds 5 2024-03-18T15:39:50.012 INFO:teuthology.orchestra.run.smithi096.stdout:in 0,1,2,3,4 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:up {0=24493,1=24485,2=24472,3=24460,4=24452} 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:failed 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:damaged 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:stopped 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:data_pools [3] 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:metadata_pool 2 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:inline_data disabled 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:balancer 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:bal_rank_mask -1 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:standby_count_wanted 0 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:qdb_cluster leader: 24493 members: 24460,24452,24485,24472,24493 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.i{0:24493} state up:active seq 5 addr [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.e{0:24422} state up:standby-replay seq 1 addr [v2:172.21.15.137:6834/3504042144,v1:172.21.15.137:6835/3504042144] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.013 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b{1:24485} state up:active seq 4 addr [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.l{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/3317488579,v1:172.21.15.193:6833/3317488579] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c{2:24472} state up:active seq 8 addr [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.k{2:24392} state up:standby-replay seq 1 addr [v2:172.21.15.137:6832/753223976,v1:172.21.15.137:6833/753223976] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.f{3:24460} state up:active seq 9 addr [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.96:6840/3021384858,v1:172.21.15.96:6841/3021384858] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.h{4:24452} state up:active seq 7 addr [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.96:6838/36196340,v1:172.21.15.96:6839/36196340] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:Standby daemons: 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.96:6834/1518442933,v1:172.21.15.96:6835/1518442933] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.014 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.96:6836/2891815137,v1:172.21.15.96:6837/2891815137] compat {c=[1],r=[1],i=[1fff]}] 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 msgr2=0x7f22800a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 0x7f22800a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2284007fd0 tx=0x7f2284007480 comp rx=0 tx=0).stop 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 msgr2=0x7f2294103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 0x7f2294103d20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f228c00b0d0 tx=0x7f228c007a20 comp rx=0 tx=0).stop 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 shutdown_connections 2024-03-18T15:39:50.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22800a2890 0x7f22800a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2294102da0 0x7f2294103d20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2294101200 0x7f2294105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 --2- 172.21.15.96:0/565186160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2294102400 0x7f22941037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 >> 172.21.15.96:0/565186160 conn(0x7f22940fc7a0 msgr2=0x7f22940fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 shutdown_connections 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:50.012+0000 7f229b479700 1 -- 172.21.15.96:0/565186160 wait complete. 2024-03-18T15:39:50.016 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:50.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-18T15:39:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:50 smithi137 ceph-mon[35868]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:50 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/565186160' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:39:50.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:50 smithi193 ceph-mon[36702]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:50 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/565186160' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:39:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:50 smithi096 ceph-mon[31930]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:39:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:50 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/565186160' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-18T15:39:50.988 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:52.888 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:52 smithi096 ceph-mon[31930]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:52.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 -- 172.21.15.96:0/587882489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102da0 msgr2=0x7feb24109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:52.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 --2- 172.21.15.96:0/587882489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102da0 0x7feb24109650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00b0d0 tx=0x7feb1c00b3e0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 -- 172.21.15.96:0/587882489 shutdown_connections 2024-03-18T15:39:52.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 --2- 172.21.15.96:0/587882489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102da0 0x7feb24109650 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 --2- 172.21.15.96:0/587882489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb24101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 --2- 172.21.15.96:0/587882489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102400 0x7feb24102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 -- 172.21.15.96:0/587882489 >> 172.21.15.96:0/587882489 conn(0x7feb240fc7a0 msgr2=0x7feb240febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 -- 172.21.15.96:0/587882489 shutdown_connections 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.898+0000 7feb2a03b700 1 -- 172.21.15.96:0/587882489 wait complete. 2024-03-18T15:39:52.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 Processor -- start 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 -- start start 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102da0 0x7feb2419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb2410cde0 con 0x7feb24102400 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feb2410cc60 con 0x7feb24101200 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb2a03b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb2410cf60 con 0x7feb24102da0 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb22ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb22ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41744/0 (socket says 172.21.15.96:41744) 2024-03-18T15:39:52.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb237fe700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb22ffd700 1 -- 172.21.15.96:0/2104600866 learned_addr learned my addr 172.21.15.96:0/2104600866 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.899+0000 7feb23fff700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102da0 0x7feb2419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 -- 172.21.15.96:0/2104600866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102da0 msgr2=0x7feb2419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102da0 0x7feb2419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 -- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 msgr2=0x7feb24197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 -- 172.21.15.96:0/2104600866 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2419f030 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb22ffd700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb237fe700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb241973d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e9c0 tx=0x7feb1800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1800cb70 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2419f320 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb1800ee60 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18018660 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.900+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2419f860 con 0x7feb24101200 2024-03-18T15:39:52.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.901+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb24068f90 con 0x7feb24101200 2024-03-18T15:39:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.902+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7feb1801f020 con 0x7feb24101200 2024-03-18T15:39:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.902+0000 7feb20ff9700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 0x7feb0c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.903+0000 7feb22ffd700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 0x7feb0c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.903+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feb180cea30 con 0x7feb24101200 2024-03-18T15:39:52.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.903+0000 7feb22ffd700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 0x7feb0c0a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb14011550 tx=0x7feb14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:52.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:52.906+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb1809ba60 con 0x7feb24101200 2024-03-18T15:39:52.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:52 smithi193 ceph-mon[36702]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:53.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.100+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7feb24062ee0 con 0x7feb24101200 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.101+0000 7feb20ff9700 1 -- 172.21.15.96:0/2104600866 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5295 (secure 0 0 0) 0x7feb1809b360 con 0x7feb24101200 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 88 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stdout:fsid 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-03-18T15:30:01.553154+0000 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-03-18T15:39:02.562128+0000 2024-03-18T15:39:53.102 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 26 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client luminous 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 12 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6802/3832630015,v1:172.21.15.96:6803/3832630015] [v2:172.21.15.96:6804/3832630015,v1:172.21.15.96:6805/3832630015] exists,up 09dd9fb0-b2af-41c6-bf5b-8c7c01d82677 2024-03-18T15:39:53.103 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/3508407501,v1:172.21.15.137:6801/3508407501] [v2:172.21.15.137:6802/3508407501,v1:172.21.15.137:6803/3508407501] exists,up 72c98ebb-ace5-408f-b919-a69839b4c3b6 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/110382863,v1:172.21.15.193:6801/110382863] [v2:172.21.15.193:6802/110382863,v1:172.21.15.193:6803/110382863] exists,up 0139a743-3058-40f5-813f-87cbe0a4d29f 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6810/157229437,v1:172.21.15.96:6811/157229437] [v2:172.21.15.96:6812/157229437,v1:172.21.15.96:6813/157229437] exists,up 8e02c4cc-033c-4b5f-944c-18fbf2ef29bf 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/1772517973,v1:172.21.15.137:6809/1772517973] [v2:172.21.15.137:6810/1772517973,v1:172.21.15.137:6811/1772517973] exists,up d1281482-ee35-4d0d-be39-618d007e714d 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/298847477,v1:172.21.15.193:6809/298847477] [v2:172.21.15.193:6810/298847477,v1:172.21.15.193:6811/298847477] exists,up a6e210dc-8cbf-4703-adec-d23138828e11 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6818/3660469146,v1:172.21.15.96:6819/3660469146] [v2:172.21.15.96:6820/3660469146,v1:172.21.15.96:6821/3660469146] exists,up dd549f23-4564-43ce-8edb-4d5cfad31277 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/1840494271,v1:172.21.15.137:6817/1840494271] [v2:172.21.15.137:6818/1840494271,v1:172.21.15.137:6819/1840494271] exists,up 01fdc4fe-0b84-4109-a831-2b38ba820711 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/1156737532,v1:172.21.15.193:6817/1156737532] [v2:172.21.15.193:6818/1156737532,v1:172.21.15.193:6819/1156737532] exists,up db47c340-59a5-4202-8488-b5548f9d18fc 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6826/205742436,v1:172.21.15.96:6827/205742436] [v2:172.21.15.96:6828/205742436,v1:172.21.15.96:6829/205742436] exists,up 214915c3-c5ad-42b8-8c12-04ecc68c6109 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/293078987,v1:172.21.15.137:6825/293078987] [v2:172.21.15.137:6826/293078987,v1:172.21.15.137:6827/293078987] exists,up 0d8d5301-49b5-4393-beb1-a3e682d035c9 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/97980715,v1:172.21.15.193:6825/97980715] [v2:172.21.15.193:6826/97980715,v1:172.21.15.193:6827/97980715] exists,up 09d05136-56ba-4ef6-be49-85d156e3c45a 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.b [2,4] 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.f [8,11] 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.2 [5,7] 2024-03-18T15:39:53.104 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.e [4,8] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.17 [4,8,5,6] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.1d [4,6] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.1e [3,9] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.22 [3,9] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.2c [4,9] 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2777887346 expires 2024-03-19T15:31:01.360993+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1517993036 expires 2024-03-19T15:31:01.360993+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/2602375712 expires 2024-03-19T15:31:01.360993+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2437645187 expires 2024-03-19T15:31:01.360993+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2398466384 expires 2024-03-19T15:30:34.844871+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/3477103440 expires 2024-03-19T15:30:34.844871+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/392915825 expires 2024-03-19T15:30:34.844871+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2538163559 expires 2024-03-19T15:30:34.844871+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/2602375712 expires 2024-03-19T15:31:01.360993+0000 2024-03-18T15:39:53.105 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/392915825 expires 2024-03-19T15:30:34.844871+0000 2024-03-18T15:39:53.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:52 smithi137 ceph-mon[35868]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 msgr2=0x7feb0c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 0x7feb0c0a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb14011550 tx=0x7feb14009260 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 msgr2=0x7feb241973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb241973d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb1800e9c0 tx=0x7feb1800ecd0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 shutdown_connections 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb0c0a2890 0x7feb0c0a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb24102da0 0x7feb2419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb24101200 0x7feb241973d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 --2- 172.21.15.96:0/2104600866 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb24102400 0x7feb24197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 >> 172.21.15.96:0/2104600866 conn(0x7feb240fc7a0 msgr2=0x7feb24105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 shutdown_connections 2024-03-18T15:39:53.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.103+0000 7feb2a03b700 1 -- 172.21.15.96:0/2104600866 wait complete. 2024-03-18T15:39:53.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:53 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2104600866' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T15:39:53.786 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-18T15:39:53.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 -- 172.21.15.96:0/147210089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181022b0 msgr2=0x7f3918108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 --2- 172.21.15.96:0/147210089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181022b0 0x7f3918108b40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3910009ca0 tx=0x7f391002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 -- 172.21.15.96:0/147210089 shutdown_connections 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 --2- 172.21.15.96:0/147210089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181022b0 0x7f3918108b40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 --2- 172.21.15.96:0/147210089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39181018f0 0x7f3918101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 --2- 172.21.15.96:0/147210089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3918100700 0x7f3918100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 -- 172.21.15.96:0/147210089 >> 172.21.15.96:0/147210089 conn(0x7f39180fbc70 msgr2=0x7f39180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 -- 172.21.15.96:0/147210089 shutdown_connections 2024-03-18T15:39:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.903+0000 7f391fe44640 1 -- 172.21.15.96:0/147210089 wait complete. 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 Processor -- start 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 -- start start 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 0x7f39180694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39181022b0 0x7f391819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f391810c360 con 0x7f39181018f0 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f391810c1e0 con 0x7f3918100700 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391fe44640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f391810c4e0 con 0x7f39181022b0 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53630/0 (socket says 172.21.15.96:53630) 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 -- 172.21.15.96:0/2519323231 learned_addr learned my addr 172.21.15.96:0/2519323231 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391dbe0640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 0x7f39180694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391e3e1640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39181022b0 0x7f391819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 -- 172.21.15.96:0/2519323231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39181022b0 msgr2=0x7f391819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39181022b0 0x7f391819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 -- 172.21.15.96:0/2519323231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 msgr2=0x7f39180694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 0x7f39180694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391d3df640 1 -- 172.21.15.96:0/2519323231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391819f720 con 0x7f39181018f0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.904+0000 7f391dbe0640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 0x7f39180694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f391d3df640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f390800e9f0 tx=0x7f390800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800cd80 con 0x7f39181018f0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3908010910 con 0x7f39181018f0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391819f9b0 con 0x7f39181018f0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908018990 con 0x7f39181018f0 2024-03-18T15:39:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.905+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391819ff00 con 0x7f39181018f0 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.906+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3918107a20 con 0x7f39181018f0 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.907+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3908010ac0 con 0x7f39181018f0 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.907+0000 7f390effd640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 0x7f39040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.907+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f39080cf190 con 0x7f39181018f0 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.907+0000 7f391dbe0640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 0x7f39040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.908+0000 7f391dbe0640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 0x7f39040a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3918101750 tx=0x7f3914005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:53.915+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f390809bbd0 con 0x7f39181018f0 2024-03-18T15:39:53.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:53 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2104600866' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T15:39:54.006 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:53 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2104600866' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-18T15:39:54.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.098+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f391802d2d0 con 0x7f39181018f0 2024-03-18T15:39:54.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.100+0000 7f390effd640 1 -- 172.21.15.96:0/2519323231 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7f390809fc70 con 0x7f39181018f0 2024-03-18T15:39:54.101 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:54.101 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:54.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.102+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 msgr2=0x7f39040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.102+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 0x7f39040a4cc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3918101750 tx=0x7f3914005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.102+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 msgr2=0x7f391819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.102+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f390800e9f0 tx=0x7f390800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.102+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 shutdown_connections 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f39040a2800 0x7f39040a4cc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39181022b0 0x7f391819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39181018f0 0x7f391819c7f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 --2- 172.21.15.96:0/2519323231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3918100700 0x7f39180694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 >> 172.21.15.96:0/2519323231 conn(0x7f39180fbc70 msgr2=0x7f3918104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 shutdown_connections 2024-03-18T15:39:54.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.103+0000 7f391fe44640 1 -- 172.21.15.96:0/2519323231 wait complete. 2024-03-18T15:39:54.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:39:54.115 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:39:54.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 -- 172.21.15.96:0/251326984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c102290 msgr2=0x7f224c108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 --2- 172.21.15.96:0/251326984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c102290 0x7f224c108b20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2244009ca0 tx=0x7f224402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 -- 172.21.15.96:0/251326984 shutdown_connections 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 --2- 172.21.15.96:0/251326984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c102290 0x7f224c108b20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 --2- 172.21.15.96:0/251326984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c1018d0 0x7f224c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 --2- 172.21.15.96:0/251326984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 -- 172.21.15.96:0/251326984 >> 172.21.15.96:0/251326984 conn(0x7f224c0fbc70 msgr2=0x7f224c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 -- 172.21.15.96:0/251326984 shutdown_connections 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.220+0000 7f2253fa9640 1 -- 172.21.15.96:0/251326984 wait complete. 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 Processor -- start 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 -- start start 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c102290 0x7f224c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10d480 con 0x7f224c1018d0 2024-03-18T15:39:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10d300 con 0x7f224c1006e0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2253fa9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10d600 con 0x7f224c102290 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2251544640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2251544640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53654/0 (socket says 172.21.15.96:53654) 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2251d45640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2251544640 1 -- 172.21.15.96:0/2599347108 learned_addr learned my addr 172.21.15.96:0/2599347108 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.221+0000 7f2252546640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c102290 0x7f224c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 -- 172.21.15.96:0/2599347108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c102290 msgr2=0x7f224c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c102290 0x7f224c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 -- 172.21.15.96:0/2599347108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 msgr2=0x7f224c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 -- 172.21.15.96:0/2599347108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c19f720 con 0x7f224c1018d0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251d45640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2251544640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9f0 tx=0x7f223c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c00cd80 con 0x7f224c1018d0 2024-03-18T15:39:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c19fa10 con 0x7f224c1018d0 2024-03-18T15:39:54.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f223c005500 con 0x7f224c1018d0 2024-03-18T15:39:54.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c010690 con 0x7f224c1018d0 2024-03-18T15:39:54.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.222+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c19ff60 con 0x7f224c1018d0 2024-03-18T15:39:54.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.223+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c107a00 con 0x7f224c1018d0 2024-03-18T15:39:54.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.224+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f223c0046c0 con 0x7f224c1018d0 2024-03-18T15:39:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.225+0000 7f2242ffd640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 0x7f22380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.225+0000 7f2251d45640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 0x7f22380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.225+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f223c0ceff0 con 0x7f224c1018d0 2024-03-18T15:39:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.225+0000 7f2251d45640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 0x7f22380a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f224c101730 tx=0x7f2248005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.232+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f223c053740 con 0x7f224c1018d0 2024-03-18T15:39:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.414+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f224c02d1b0 con 0x7f224c1018d0 2024-03-18T15:39:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.414+0000 7f2242ffd640 1 -- 172.21.15.96:0/2599347108 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f223c09c120 con 0x7f224c1018d0 2024-03-18T15:39:54.415 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:39:54.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 msgr2=0x7f22380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 0x7f22380a4cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f224c101730 tx=0x7f2248005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:54.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 msgr2=0x7f224c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f223c00e9f0 tx=0x7f223c00edf0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 shutdown_connections 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2800 0x7f22380a4cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c102290 0x7f224c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c1018d0 0x7f224c198040 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 --2- 172.21.15.96:0/2599347108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1006e0 0x7f224c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.416+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 >> 172.21.15.96:0/2599347108 conn(0x7f224c0fbc70 msgr2=0x7f224c104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.417+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 shutdown_connections 2024-03-18T15:39:54.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.417+0000 7f2253fa9640 1 -- 172.21.15.96:0/2599347108 wait complete. 2024-03-18T15:39:54.429 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.532+0000 7f576440c640 1 -- 172.21.15.96:0/15402061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c1018d0 msgr2=0x7f575c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.532+0000 7f576440c640 1 --2- 172.21.15.96:0/15402061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c1018d0 0x7f575c101d50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f574c009900 tx=0x7f574c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- 172.21.15.96:0/15402061 shutdown_connections 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- 172.21.15.96:0/15402061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c102290 0x7f575c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- 172.21.15.96:0/15402061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c1018d0 0x7f575c101d50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- 172.21.15.96:0/15402061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 0x7f575c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- 172.21.15.96:0/15402061 >> 172.21.15.96:0/15402061 conn(0x7f575c0fbc70 msgr2=0x7f575c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- 172.21.15.96:0/15402061 shutdown_connections 2024-03-18T15:39:54.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- 172.21.15.96:0/15402061 wait complete. 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 Processor -- start 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- start start 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 0x7f575c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c1018d0 0x7f575c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10d480 con 0x7f575c102290 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10d300 con 0x7f575c1006e0 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f576440c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f575c10d600 con 0x7f575c1018d0 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f57629a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f57629a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53678/0 (socket says 172.21.15.96:53678) 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f57629a9640 1 -- 172.21.15.96:0/3240052968 learned_addr learned my addr 172.21.15.96:0/3240052968 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f57621a8640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 0x7f575c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.533+0000 7f57619a7640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c1018d0 0x7f575c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 -- 172.21.15.96:0/3240052968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c1018d0 msgr2=0x7f575c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c1018d0 0x7f575c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 -- 172.21.15.96:0/3240052968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 msgr2=0x7f575c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 0x7f575c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 -- 172.21.15.96:0/3240052968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c19f710 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57629a9640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f575800e970 tx=0x7f575800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575800ccc0 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f575800ce60 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c19fa00 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758010490 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.534+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c19ff20 con 0x7f575c102290 2024-03-18T15:39:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.535+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c107aa0 con 0x7f575c102290 2024-03-18T15:39:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.536+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5758005000 con 0x7f575c102290 2024-03-18T15:39:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.536+0000 7f57537fe640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 0x7f57480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.537+0000 7f57621a8640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 0x7f57480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.537+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57580cf000 con 0x7f575c102290 2024-03-18T15:39:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.537+0000 7f57621a8640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 0x7f57480a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f575c101730 tx=0x7f5754009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.540+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5758053740 con 0x7f575c102290 2024-03-18T15:39:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:54 smithi096 ceph-mon[31930]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:54 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2519323231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:54 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2599347108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:54.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.716+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f575c02d200 con 0x7f575c102290 2024-03-18T15:39:54.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.716+0000 7f57537fe640 1 -- 172.21.15.96:0/3240052968 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f575809c130 con 0x7f575c102290 2024-03-18T15:39:54.718 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 msgr2=0x7f57480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 0x7f57480a4cc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f575c101730 tx=0x7f5754009500 comp rx=0 tx=0).stop 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 msgr2=0x7f575c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f575800e970 tx=0x7f575800ed70 comp rx=0 tx=0).stop 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 shutdown_connections 2024-03-18T15:39:54.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57480a2800 0x7f57480a4cc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f575c102290 0x7f575c19f0a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f575c1018d0 0x7f575c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 --2- 172.21.15.96:0/3240052968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f575c1006e0 0x7f575c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 >> 172.21.15.96:0/3240052968 conn(0x7f575c0fbc70 msgr2=0x7f575c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 shutdown_connections 2024-03-18T15:39:54.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.718+0000 7f576440c640 1 -- 172.21.15.96:0/3240052968 wait complete. 2024-03-18T15:39:54.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.833+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/2255499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 msgr2=0x7f0bc4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.833+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/2255499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4101d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009900 tx=0x7f0bb402bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/2255499451 shutdown_connections 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/2255499451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc4102290 0x7f0bc4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/2255499451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4101d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/2255499451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc41006e0 0x7f0bc4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/2255499451 >> 172.21.15.96:0/2255499451 conn(0x7f0bc40fbc70 msgr2=0x7f0bc40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/2255499451 shutdown_connections 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/2255499451 wait complete. 2024-03-18T15:39:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 Processor -- start 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- start start 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc4102290 0x7f0bc419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410d470 con 0x7f0bc41006e0 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410d2f0 con 0x7f0bc41018d0 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcc4d1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc410d5f0 con 0x7f0bc4102290 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bca26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bca26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53704/0 (socket says 172.21.15.96:53704) 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bca26d640 1 -- 172.21.15.96:0/1873389394 learned_addr learned my addr 172.21.15.96:0/1873389394 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.834+0000 7f0bcaa6e640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc4102290 0x7f0bc419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bc9a6c640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 -- 172.21.15.96:0/1873389394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc4102290 msgr2=0x7f0bc419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc4102290 0x7f0bc419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 -- 172.21.15.96:0/1873389394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 msgr2=0x7f0bc4198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 -- 172.21.15.96:0/1873389394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc419f6f0 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bca26d640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000e9c0 tx=0x7f0bc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000cd80 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc419f9e0 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc00035f0 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0010690 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.835+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc419ff50 con 0x7f0bc41006e0 2024-03-18T15:39:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.836+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc4107a20 con 0x7f0bc41006e0 2024-03-18T15:39:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.837+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f0bc00037a0 con 0x7f0bc41006e0 2024-03-18T15:39:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.837+0000 7f0bbb7fe640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.838+0000 7f0bc9a6c640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.838+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f0bc00cee20 con 0x7f0bc41006e0 2024-03-18T15:39:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.838+0000 7f0bc9a6c640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0bb402c7b0 tx=0x7f0bb4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:54.842+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0bc009beb0 con 0x7f0bc41006e0 2024-03-18T15:39:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:54 smithi193 ceph-mon[36702]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:54 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2519323231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:54 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2599347108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.015+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0bc40623e0 con 0x7f0bc41006e0 2024-03-18T15:39:55.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.016+0000 7f0bbb7fe640 1 -- 172.21.15.96:0/1873389394 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f0bc009b7b0 con 0x7f0bc41006e0 2024-03-18T15:39:55.017 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 msgr2=0x7f0bb00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0bb402c7b0 tx=0x7f0bb4005fd0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 msgr2=0x7f0bc4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000e9c0 tx=0x7f0bc000ee80 comp rx=0 tx=0).stop 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 shutdown_connections 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0bb00a2800 0x7f0bb00a4cc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bc4102290 0x7f0bc419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bc41018d0 0x7f0bc4198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 --2- 172.21.15.96:0/1873389394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bc41006e0 0x7f0bc4197ad0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 >> 172.21.15.96:0/1873389394 conn(0x7f0bc40fbc70 msgr2=0x7f0bc4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 shutdown_connections 2024-03-18T15:39:55.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.018+0000 7f0bcc4d1640 1 -- 172.21.15.96:0/1873389394 wait complete. 2024-03-18T15:39:55.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:39:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:54 smithi137 ceph-mon[35868]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-18T15:39:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:54 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2519323231' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:54 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2599347108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 -- 172.21.15.96:0/478187404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 msgr2=0x7fea58101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 --2- 172.21.15.96:0/478187404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea58101d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea40009900 tx=0x7fea4002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 -- 172.21.15.96:0/478187404 shutdown_connections 2024-03-18T15:39:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 --2- 172.21.15.96:0/478187404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea581022d0 0x7fea58108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 --2- 172.21.15.96:0/478187404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea58101d90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 --2- 172.21.15.96:0/478187404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea58100720 0x7fea58100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 -- 172.21.15.96:0/478187404 >> 172.21.15.96:0/478187404 conn(0x7fea580fbc90 msgr2=0x7fea580fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 -- 172.21.15.96:0/478187404 shutdown_connections 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.130+0000 7fea5de34640 1 -- 172.21.15.96:0/478187404 wait complete. 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 Processor -- start 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 -- start start 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea58100720 0x7fea5806aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea581022d0 0x7fea5819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810c3a0 con 0x7fea58101910 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810c220 con 0x7fea581022d0 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea5de34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea5810c520 con 0x7fea58100720 2024-03-18T15:39:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53710/0 (socket says 172.21.15.96:53710) 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 -- 172.21.15.96:0/2437279463 learned_addr learned my addr 172.21.15.96:0/2437279463 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea577fe640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea58100720 0x7fea5806aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea57fff640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea581022d0 0x7fea5819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 -- 172.21.15.96:0/2437279463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea58100720 msgr2=0x7fea5806aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea58100720 0x7fea5806aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 -- 172.21.15.96:0/2437279463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea581022d0 msgr2=0x7fea5819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea581022d0 0x7fea5819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.131+0000 7fea4effd640 1 -- 172.21.15.96:0/2437279463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819f770 con 0x7fea58101910 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea4effd640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea4002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40041070 con 0x7fea58101910 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5819fa00 con 0x7fea58101910 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40004450 con 0x7fea58101910 2024-03-18T15:39:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40003e20 con 0x7fea58101910 2024-03-18T15:39:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.132+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5819ff40 con 0x7fea58101910 2024-03-18T15:39:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.133+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea58107a40 con 0x7fea58101910 2024-03-18T15:39:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.134+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fea40044070 con 0x7fea58101910 2024-03-18T15:39:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.134+0000 7fea557fa640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 0x7fea440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.135+0000 7fea577fe640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 0x7fea440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.135+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fea400f0940 con 0x7fea58101910 2024-03-18T15:39:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.135+0000 7fea577fe640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 0x7fea440a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea58101770 tx=0x7fea48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.138+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fea400bd380 con 0x7fea58101910 2024-03-18T15:39:55.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.353+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5802d1b0 con 0x7fea58101910 2024-03-18T15:39:55.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.354+0000 7fea557fa640 1 -- 172.21.15.96:0/2437279463 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7fea400c1420 con 0x7fea58101910 2024-03-18T15:39:55.355 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:55.355 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:39:55.355 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 msgr2=0x7fea440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 0x7fea440a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fea58101770 tx=0x7fea48005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 msgr2=0x7fea5819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea40009230 tx=0x7fea4002c650 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 shutdown_connections 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fea440a2800 0x7fea440a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea581022d0 0x7fea5819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea58101910 0x7fea5819c870 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 --2- 172.21.15.96:0/2437279463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea58100720 0x7fea5806aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 >> 172.21.15.96:0/2437279463 conn(0x7fea580fbc90 msgr2=0x7fea58104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 shutdown_connections 2024-03-18T15:39:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.356+0000 7fea5de34640 1 -- 172.21.15.96:0/2437279463 wait complete. 2024-03-18T15:39:55.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/3463098804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 msgr2=0x7fe0a8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/3463098804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8102e80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe090009900 tx=0x7fe09002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/3463098804 shutdown_connections 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/3463098804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a81033c0 0x7fe0a8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/3463098804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8102e80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/3463098804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a8101810 0x7fe0a8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/3463098804 >> 172.21.15.96:0/3463098804 conn(0x7fe0a80fcdc0 msgr2=0x7fe0a80ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/3463098804 shutdown_connections 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.467+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/3463098804 wait complete. 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 Processor -- start 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 -- start start 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 0x7fe0a8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a81033c0 0x7fe0a819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a810d490 con 0x7fe0a8102a00 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a810d310 con 0x7fe0a8101810 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0ad3f3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a810d610 con 0x7fe0a81033c0 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53736/0 (socket says 172.21.15.96:53736) 2024-03-18T15:39:55.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 -- 172.21.15.96:0/2792624185 learned_addr learned my addr 172.21.15.96:0/2792624185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0a6ffd640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 0x7fe0a8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0a77fe640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a81033c0 0x7fe0a819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 -- 172.21.15.96:0/2792624185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a81033c0 msgr2=0x7fe0a819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a81033c0 0x7fe0a819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 -- 172.21.15.96:0/2792624185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 msgr2=0x7fe0a8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 0x7fe0a8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe09ffff640 1 -- 172.21.15.96:0/2792624185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a819f760 con 0x7fe0a8102a00 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.468+0000 7fe0a6ffd640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 0x7fe0a8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe09ffff640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe090009230 tx=0x7fe09002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090041070 con 0x7fe0a8102a00 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe090004450 con 0x7fe0a8102a00 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a819f9f0 con 0x7fe0a8102a00 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090003e20 con 0x7fe0a8102a00 2024-03-18T15:39:55.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.469+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a819fee0 con 0x7fe0a8102a00 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.470+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a8108b30 con 0x7fe0a8102a00 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.471+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fe090044070 con 0x7fe0a8102a00 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.472+0000 7fe0a4ff9640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 0x7fe0940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.472+0000 7fe0a6ffd640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 0x7fe0940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.472+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fe0900f0970 con 0x7fe0a8102a00 2024-03-18T15:39:55.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.472+0000 7fe0a6ffd640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 0x7fe0940a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8102860 tx=0x7fe098005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.476+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe0900bd3b0 con 0x7fe0a8102a00 2024-03-18T15:39:55.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.652+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0a802d400 con 0x7fe0a8102a00 2024-03-18T15:39:55.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.654+0000 7fe0a4ff9640 1 -- 172.21.15.96:0/2792624185 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7fe0900c1450 con 0x7fe0a8102a00 2024-03-18T15:39:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:55.655 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 msgr2=0x7fe0940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 0x7fe0940a4cc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8102860 tx=0x7fe098005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 msgr2=0x7fe0a8198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe090009230 tx=0x7fe09002c650 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 shutdown_connections 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0940a2800 0x7fe0940a4cc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0a81033c0 0x7fe0a819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0a8102a00 0x7fe0a8198080 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 --2- 172.21.15.96:0/2792624185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0a8101810 0x7fe0a8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 >> 172.21.15.96:0/2792624185 conn(0x7fe0a80fcdc0 msgr2=0x7fe0a8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 shutdown_connections 2024-03-18T15:39:55.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.656+0000 7fe0ad3f3640 1 -- 172.21.15.96:0/2792624185 wait complete. 2024-03-18T15:39:55.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:55 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3240052968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:55 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1873389394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.666 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:55 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2437279463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:55.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:39:55.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 -- 172.21.15.96:0/95811439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b081006e0 msgr2=0x7f3b08100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/95811439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b081006e0 0x7f3b08100b00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ca0 tx=0x7f3b0402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 -- 172.21.15.96:0/95811439 shutdown_connections 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/95811439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b08102290 0x7f3b08108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/95811439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081018d0 0x7f3b08101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/95811439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b081006e0 0x7f3b08100b00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.771+0000 7f3b0fccc640 1 -- 172.21.15.96:0/95811439 >> 172.21.15.96:0/95811439 conn(0x7f3b080fbc70 msgr2=0x7f3b080fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- 172.21.15.96:0/95811439 shutdown_connections 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- 172.21.15.96:0/95811439 wait complete. 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 Processor -- start 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- start start 2024-03-18T15:39:55.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 0x7f3b08197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b081018d0 0x7f3b08198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0810c360 con 0x7f3b08102290 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0810c1e0 con 0x7f3b081006e0 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0fccc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0810c4e0 con 0x7f3b081018d0 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0e269640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0e269640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53758/0 (socket says 172.21.15.96:53758) 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0e269640 1 -- 172.21.15.96:0/3703022297 learned_addr learned my addr 172.21.15.96:0/3703022297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0da68640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 0x7f3b08197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.772+0000 7f3b0d267640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b081018d0 0x7f3b08198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 -- 172.21.15.96:0/3703022297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b081018d0 msgr2=0x7f3b08198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b081018d0 0x7f3b08198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 -- 172.21.15.96:0/3703022297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 msgr2=0x7f3b08197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 0x7f3b08197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 -- 172.21.15.96:0/3703022297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0819f730 con 0x7f3b08102290 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0da68640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 0x7f3b08197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:55.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0e269640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000e970 tx=0x7f3b0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0000ccc0 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0000ce60 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0819fa20 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00010490 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.773+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0819ffc0 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.774+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b081a0250 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.775+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f3b00005020 con 0x7f3b08102290 2024-03-18T15:39:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.776+0000 7f3afeffd640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 0x7f3af40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.776+0000 7f3b0da68640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 0x7f3af40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.776+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f3b000cedf0 con 0x7f3b08102290 2024-03-18T15:39:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.776+0000 7f3b0da68640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 0x7f3af40a4d40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c700 tx=0x7f3b04032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.780+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3b0009be80 con 0x7f3b08102290 2024-03-18T15:39:55.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:55 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3240052968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:55 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1873389394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:55 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2437279463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:55.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.956+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3b081a0420 con 0x7f3b08102290 2024-03-18T15:39:55.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.956+0000 7f3afeffd640 1 -- 172.21.15.96:0/3703022297 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f3b0009b780 con 0x7f3b08102290 2024-03-18T15:39:55.957 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:39:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 msgr2=0x7f3af40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 0x7f3af40a4d40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3b0402c700 tx=0x7f3b04032590 comp rx=0 tx=0).stop 2024-03-18T15:39:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 msgr2=0x7f3b0819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:55.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3b0000e970 tx=0x7f3b0000ed70 comp rx=0 tx=0).stop 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 shutdown_connections 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af40a2880 0x7f3af40a4d40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b08102290 0x7f3b0819f0c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b081018d0 0x7f3b08198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 --2- 172.21.15.96:0/3703022297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b081006e0 0x7f3b08197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 >> 172.21.15.96:0/3703022297 conn(0x7f3b080fbc70 msgr2=0x7f3b08068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 shutdown_connections 2024-03-18T15:39:55.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:55.958+0000 7f3b0fccc640 1 -- 172.21.15.96:0/3703022297 wait complete. 2024-03-18T15:39:55.970 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 -- 172.21.15.96:0/3700081132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60441018d0 msgr2=0x7f6044101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 --2- 172.21.15.96:0/3700081132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60441018d0 0x7f6044101d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 -- 172.21.15.96:0/3700081132 shutdown_connections 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 --2- 172.21.15.96:0/3700081132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6044102290 0x7f6044108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 --2- 172.21.15.96:0/3700081132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60441018d0 0x7f6044101d50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 --2- 172.21.15.96:0/3700081132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f6044100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 -- 172.21.15.96:0/3700081132 >> 172.21.15.96:0/3700081132 conn(0x7f60440fbc70 msgr2=0x7f60440fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 -- 172.21.15.96:0/3700081132 shutdown_connections 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.073+0000 7f6049377640 1 -- 172.21.15.96:0/3700081132 wait complete. 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 Processor -- start 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 -- start start 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f60440694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 0x7f604419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f604410c360 con 0x7f6044102290 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f604410c1e0 con 0x7f60441018d0 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6049377640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f604410c4e0 con 0x7f60441006e0 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f60437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f60437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53772/0 (socket says 172.21.15.96:53772) 2024-03-18T15:39:56.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f60437fe640 1 -- 172.21.15.96:0/77507477 learned_addr learned my addr 172.21.15.96:0/77507477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f6042ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f60440694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.074+0000 7f60427fc640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 0x7f604419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 -- 172.21.15.96:0/77507477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 msgr2=0x7f60440694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f60440694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 -- 172.21.15.96:0/77507477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 msgr2=0x7f604419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 0x7f604419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 -- 172.21.15.96:0/77507477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419f750 con 0x7f6044102290 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f6042ffd640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f60440694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60427fc640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 0x7f604419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f60437fe640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e970 tx=0x7f603c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c00ccc0 con 0x7f6044102290 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f603c00ce60 con 0x7f6044102290 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419fa40 con 0x7f6044102290 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c010490 con 0x7f6044102290 2024-03-18T15:39:56.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.075+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604419ff10 con 0x7f6044102290 2024-03-18T15:39:56.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.076+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044107ad0 con 0x7f6044102290 2024-03-18T15:39:56.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.077+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f603c005020 con 0x7f6044102290 2024-03-18T15:39:56.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.077+0000 7f602bfff640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 0x7f602c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.077+0000 7f6042ffd640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 0x7f602c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.077+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f603c0ceee0 con 0x7f6044102290 2024-03-18T15:39:56.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.078+0000 7f6042ffd640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 0x7f602c0a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6044101730 tx=0x7f6034009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.082+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f603c09bf70 con 0x7f6044102290 2024-03-18T15:39:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:55 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3240052968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:55 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1873389394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:55 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2437279463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:56.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.258+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60440623e0 con 0x7f6044102290 2024-03-18T15:39:56.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.258+0000 7f602bfff640 1 -- 172.21.15.96:0/77507477 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f603c09b870 con 0x7f6044102290 2024-03-18T15:39:56.259 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:39:56.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 msgr2=0x7f602c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 0x7f602c0a4cc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6044101730 tx=0x7f6034009500 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 msgr2=0x7f604419ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e970 tx=0x7f603c00ed70 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 shutdown_connections 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f602c0a2800 0x7f602c0a4cc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6044102290 0x7f604419ee90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60441018d0 0x7f604419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 --2- 172.21.15.96:0/77507477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60441006e0 0x7f60440694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.260+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 >> 172.21.15.96:0/77507477 conn(0x7f60440fbc70 msgr2=0x7f6044104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.261+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 shutdown_connections 2024-03-18T15:39:56.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.261+0000 7f6049377640 1 -- 172.21.15.96:0/77507477 wait complete. 2024-03-18T15:39:56.273 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- 172.21.15.96:0/2813004148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 msgr2=0x7f54e0108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 --2- 172.21.15.96:0/2813004148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e0108b40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009ca0 tx=0x7f54d802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- 172.21.15.96:0/2813004148 shutdown_connections 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 --2- 172.21.15.96:0/2813004148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e0108b40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 --2- 172.21.15.96:0/2813004148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e01018f0 0x7f54e0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 --2- 172.21.15.96:0/2813004148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e0100700 0x7f54e0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- 172.21.15.96:0/2813004148 >> 172.21.15.96:0/2813004148 conn(0x7f54e00fbc90 msgr2=0x7f54e00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- 172.21.15.96:0/2813004148 shutdown_connections 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- 172.21.15.96:0/2813004148 wait complete. 2024-03-18T15:39:56.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 Processor -- start 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.377+0000 7f54e6285640 1 -- start start 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 0x7f54e0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e01018f0 0x7f54e0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c380 con 0x7f54e01022b0 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c200 con 0x7f54e0100700 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010c500 con 0x7f54e01018f0 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54dffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 0x7f54e0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e01018f0 0x7f54e0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53800/0 (socket says 172.21.15.96:53800) 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 -- 172.21.15.96:0/636230006 learned_addr learned my addr 172.21.15.96:0/636230006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54dffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 0x7f54e0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:36276/0 (socket says 172.21.15.96:36276) 2024-03-18T15:39:56.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 -- 172.21.15.96:0/636230006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e01018f0 msgr2=0x7f54e0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e01018f0 0x7f54e0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 -- 172.21.15.96:0/636230006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 msgr2=0x7f54e0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 0x7f54e0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 -- 172.21.15.96:0/636230006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e019f6f0 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e4822640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f54d8037040 tx=0x7f54d8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8041020 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d8034d90 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e019f980 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8039040 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.378+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e019fe70 con 0x7f54e01022b0 2024-03-18T15:39:56.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.379+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e01a00a0 con 0x7f54e01022b0 2024-03-18T15:39:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.380+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f54d8044020 con 0x7f54e01022b0 2024-03-18T15:39:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.381+0000 7f54dd7fa640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 0x7f54cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.381+0000 7f54dffff640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 0x7f54cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.381+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f54d80f0950 con 0x7f54e01022b0 2024-03-18T15:39:56.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.381+0000 7f54dffff640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 0x7f54cc0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54e0101750 tx=0x7f54c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.385+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f54d804b020 con 0x7f54e01022b0 2024-03-18T15:39:56.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.560+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54e01a0270 con 0x7f54e01022b0 2024-03-18T15:39:56.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.560+0000 7f54dd7fa640 1 -- 172.21.15.96:0/636230006 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f54d80bd9e0 con 0x7f54e01022b0 2024-03-18T15:39:56.561 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:39:56.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.562+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 msgr2=0x7f54cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.562+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 0x7f54cc0a4cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54e0101750 tx=0x7f54c8005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.562+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 msgr2=0x7f54e019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.562+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f54d8037040 tx=0x7f54d8032f20 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 shutdown_connections 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54cc0a2800 0x7f54cc0a4cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f54e01022b0 0x7f54e019f080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54e01018f0 0x7f54e0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 --2- 172.21.15.96:0/636230006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e0100700 0x7f54e0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 >> 172.21.15.96:0/636230006 conn(0x7f54e00fbc90 msgr2=0x7f54e0069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 shutdown_connections 2024-03-18T15:39:56.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.563+0000 7f54e6285640 1 -- 172.21.15.96:0/636230006 wait complete. 2024-03-18T15:39:56.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- 172.21.15.96:0/1903412579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 msgr2=0x7f5724107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 --2- 172.21.15.96:0/1903412579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724107080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5710009900 tx=0x7f571002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- 172.21.15.96:0/1903412579 shutdown_connections 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 --2- 172.21.15.96:0/1903412579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57241075c0 0x7f57241099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 --2- 172.21.15.96:0/1903412579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724107080 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 --2- 172.21.15.96:0/1903412579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5724102360 0x7f5724104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- 172.21.15.96:0/1903412579 >> 172.21.15.96:0/1903412579 conn(0x7f57240fbc90 msgr2=0x7f57240fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- 172.21.15.96:0/1903412579 shutdown_connections 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- 172.21.15.96:0/1903412579 wait complete. 2024-03-18T15:39:56.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 Processor -- start 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.676+0000 7f572ae20640 1 -- start start 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 0x7f5724101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57241075c0 0x7f5724100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f572410d4a0 con 0x7f5724104c90 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f572410d320 con 0x7f5724102360 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f572410d620 con 0x7f57241075c0 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53806/0 (socket says 172.21.15.96:53806) 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 -- 172.21.15.96:0/1580841926 learned_addr learned my addr 172.21.15.96:0/1580841926 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5728bbc640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 0x7f5724101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f57293bd640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57241075c0 0x7f5724100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 -- 172.21.15.96:0/1580841926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57241075c0 msgr2=0x7f5724100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57241075c0 0x7f5724100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 -- 172.21.15.96:0/1580841926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 msgr2=0x7f5724101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 0x7f5724101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 -- 172.21.15.96:0/1580841926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57241010e0 con 0x7f5724104c90 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5728bbc640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 0x7f5724101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5723fff640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f571002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710041070 con 0x7f5724104c90 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5710004450 con 0x7f5724104c90 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57241a1c60 con 0x7f5724104c90 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.677+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710034440 con 0x7f5724104c90 2024-03-18T15:39:56.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.678+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57241a21b0 con 0x7f5724104c90 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.679+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57240fd780 con 0x7f5724104c90 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.680+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f5710044070 con 0x7f5724104c90 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.680+0000 7f5721ffb640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 0x7f57140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.680+0000 7f5728bbc640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 0x7f57140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.681+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f57100f0680 con 0x7f5724104c90 2024-03-18T15:39:56.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.681+0000 7f5728bbc640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 0x7f57140a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5718009c20 tx=0x7f5718005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:56.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.685+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f57100bd7a0 con 0x7f5724104c90 2024-03-18T15:39:56.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.900+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f572402d1b0 con 0x7f5724104c90 2024-03-18T15:39:56.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.901+0000 7f5721ffb640 1 -- 172.21.15.96:0/1580841926 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f57100bd0a0 con 0x7f5724104c90 2024-03-18T15:39:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:56.902 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:38:11.142722+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:39:56.902 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 24 2024-03-18T15:39:56.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 msgr2=0x7f57140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 0x7f57140a4cc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5718009c20 tx=0x7f5718005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 msgr2=0x7f5724100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f571002c640 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 shutdown_connections 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f57140a2800 0x7f57140a4cc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57241075c0 0x7f5724100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5724104c90 0x7f5724100310 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.903+0000 7f572ae20640 1 --2- 172.21.15.96:0/1580841926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724102360 0x7f5724101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.904+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 >> 172.21.15.96:0/1580841926 conn(0x7f57240fbc90 msgr2=0x7f5724102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.904+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 shutdown_connections 2024-03-18T15:39:56.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:56.904+0000 7f572ae20640 1 -- 172.21.15.96:0/1580841926 wait complete. 2024-03-18T15:39:56.914 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:56 smithi096 ceph-mon[31930]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:56.914 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2792624185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:56.914 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3703022297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.914 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/77507477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.914 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:56 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/636230006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:39:56.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:56 smithi193 ceph-mon[36702]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:56 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2792624185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:56 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3703022297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:56 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/77507477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:56 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/636230006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:57.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 -- 172.21.15.96:0/2444021311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1022b0 msgr2=0x7f35ac108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 --2- 172.21.15.96:0/2444021311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1022b0 0x7f35ac108b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009ca0 tx=0x7f35a402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 -- 172.21.15.96:0/2444021311 shutdown_connections 2024-03-18T15:39:57.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 --2- 172.21.15.96:0/2444021311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1022b0 0x7f35ac108b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 --2- 172.21.15.96:0/2444021311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1018f0 0x7f35ac101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 --2- 172.21.15.96:0/2444021311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac100700 0x7f35ac100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 -- 172.21.15.96:0/2444021311 >> 172.21.15.96:0/2444021311 conn(0x7f35ac0fbc70 msgr2=0x7f35ac0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 -- 172.21.15.96:0/2444021311 shutdown_connections 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.016+0000 7f35b1b62640 1 -- 172.21.15.96:0/2444021311 wait complete. 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 Processor -- start 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 -- start start 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 0x7f35ac19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1022b0 0x7f35ac19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c380 con 0x7f35ac100700 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c200 con 0x7f35ac1018f0 2024-03-18T15:39:57.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35b1b62640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c500 con 0x7f35ac1022b0 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53822/0 (socket says 172.21.15.96:53822) 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 -- 172.21.15.96:0/1525283960 learned_addr learned my addr 172.21.15.96:0/1525283960 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35abfff640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1022b0 0x7f35ac19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35aaffd640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 0x7f35ac19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 -- 172.21.15.96:0/1525283960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1022b0 msgr2=0x7f35ac19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1022b0 0x7f35ac19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 -- 172.21.15.96:0/1525283960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 msgr2=0x7f35ac19c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 0x7f35ac19c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 -- 172.21.15.96:0/1525283960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19f6f0 con 0x7f35ac100700 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35aaffd640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 0x7f35ac19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.017+0000 7f35ab7fe640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9c0 tx=0x7f359c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.018+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00cd80 con 0x7f35ac100700 2024-03-18T15:39:57.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.018+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19f9e0 con 0x7f35ac100700 2024-03-18T15:39:57.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.018+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c0035f0 con 0x7f35ac100700 2024-03-18T15:39:57.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.018+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c010690 con 0x7f35ac100700 2024-03-18T15:39:57.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.018+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac19ff30 con 0x7f35ac100700 2024-03-18T15:39:57.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.019+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac107a20 con 0x7f35ac100700 2024-03-18T15:39:57.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.020+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f359c0037a0 con 0x7f35ac100700 2024-03-18T15:39:57.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.021+0000 7f35a8ff9640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 0x7f35940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.021+0000 7f35aaffd640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 0x7f35940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.021+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f359c0cefb0 con 0x7f35ac100700 2024-03-18T15:39:57.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.021+0000 7f35aaffd640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 0x7f35940a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35ac069240 tx=0x7f35a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.025+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f359c09c0d0 con 0x7f35ac100700 2024-03-18T15:39:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:56 smithi137 ceph-mon[35868]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:39:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:56 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2792624185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:56 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3703022297' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:39:57.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:56 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/77507477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:39:57.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:56 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/636230006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:39:57.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.200+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35ac02d370 con 0x7f35ac100700 2024-03-18T15:39:57.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.202+0000 7f35a8ff9640 1 -- 172.21.15.96:0/1525283960 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22667 (secure 0 0 0) 0x7f359c09b9d0 con 0x7f35ac100700 2024-03-18T15:39:57.203 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:39:57.203 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":88,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:39:02.562128+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 msgr2=0x7f35940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 0x7f35940a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35ac069240 tx=0x7f35a0005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 msgr2=0x7f35ac0694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9c0 tx=0x7f359c00ee80 comp rx=0 tx=0).stop 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 shutdown_connections 2024-03-18T15:39:57.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35940a2800 0x7f35940a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1022b0 0x7f35ac19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018f0 0x7f35ac19c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 --2- 172.21.15.96:0/1525283960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac100700 0x7f35ac0694d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 >> 172.21.15.96:0/1525283960 conn(0x7f35ac0fbc70 msgr2=0x7f35ac104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 shutdown_connections 2024-03-18T15:39:57.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.204+0000 7f35b1b62640 1 -- 172.21.15.96:0/1525283960 wait complete. 2024-03-18T15:39:57.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-18T15:38:04.167866+0000', 'modified': '2024-03-18T15:38:11.142722+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24493, 'mds_1': 24485, 'mds_2': 24472, 'mds_3': 24460, 'mds_4': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6839/36196340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 36196340}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 36196340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6841/3021384858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6840', 'nonce': 3021384858}, {'type': 'v1', 'addr': '172.21.15.96:6841', 'nonce': 3021384858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6833/753223976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6832', 'nonce': 753223976}, {'type': 'v1', 'addr': '172.21.15.137:6833', 'nonce': 753223976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/3317488579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 3317488579}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 3317488579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.137:6835/3504042144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6834', 'nonce': 3504042144}, {'type': 'v1', 'addr': '172.21.15.137:6835', 'nonce': 3504042144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.137:6837/2776916439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6836', 'nonce': 2776916439}, {'type': 'v1', 'addr': '172.21.15.137:6837', 'nonce': 2776916439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'f', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.193:6835/1114653826', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 1114653826}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 1114653826}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6837/2478672579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2478672579}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2478672579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.137:6839/1070718965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 1070718965}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 1070718965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.193:6839/1184009925', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 1184009925}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 1184009925}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24460, 24452, 24485, 24472, 24493]} 2024-03-18T15:39:57.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-18T15:39:57.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-18T15:39:57.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- 172.21.15.96:0/525528684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb070101800 msgr2=0x7fb070101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 --2- 172.21.15.96:0/525528684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb070101800 0x7fb070101c20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb060009900 tx=0x7fb06002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- 172.21.15.96:0/525528684 shutdown_connections 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 --2- 172.21.15.96:0/525528684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701033b0 0x7fb070109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 --2- 172.21.15.96:0/525528684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0701029f0 0x7fb070102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 --2- 172.21.15.96:0/525528684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb070101800 0x7fb070101c20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- 172.21.15.96:0/525528684 >> 172.21.15.96:0/525528684 conn(0x7fb0700fcd50 msgr2=0x7fb0700ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- 172.21.15.96:0/525528684 shutdown_connections 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- 172.21.15.96:0/525528684 wait complete. 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 Processor -- start 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.319+0000 7fb075ac6640 1 -- start start 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070101800 0x7fb070197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701029f0 0x7fb070198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb07010d480 con 0x7fb0701033b0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb07010d300 con 0x7fb070101800 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb07010d600 con 0x7fb0701029f0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53840/0 (socket says 172.21.15.96:53840) 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 -- 172.21.15.96:0/2234453695 learned_addr learned my addr 172.21.15.96:0/2234453695 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070101800 0x7fb070197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701029f0 0x7fb070198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 -- 172.21.15.96:0/2234453695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701029f0 msgr2=0x7fb070198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701029f0 0x7fb070198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 -- 172.21.15.96:0/2234453695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070101800 msgr2=0x7fb070197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070101800 0x7fb070197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 -- 172.21.15.96:0/2234453695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07019f730 con 0x7fb0701033b0 2024-03-18T15:39:57.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06ffff640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb06800eb10 tx=0x7fb06800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06d7fa640 1 -- 172.21.15.96:0/2234453695 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068004b20 con 0x7fb0701033b0 2024-03-18T15:39:57.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07019fa20 con 0x7fb0701033b0 2024-03-18T15:39:57.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06d7fa640 1 -- 172.21.15.96:0/2234453695 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb068004cc0 con 0x7fb0701033b0 2024-03-18T15:39:57.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb06d7fa640 1 -- 172.21.15.96:0/2234453695 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068010760 con 0x7fb0701033b0 2024-03-18T15:39:57.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.320+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07019ff70 con 0x7fb0701033b0 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.322+0000 7fb06d7fa640 1 -- 172.21.15.96:0/2234453695 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7fb068005020 con 0x7fb0701033b0 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 Processor -- start 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 -- start start 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 0x55887852f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852f730 0x558878531bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55887848dc80 con 0x558878529bd0 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558878436aa0 con 0x55887852a5c0 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb06d7fa640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 0x7fb05c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb07878c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558878489d20 con 0x55887852f730 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53848/0 (socket says 172.21.15.96:53848) 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 -- 172.21.15.96:0/2277448706 learned_addr learned my addr 172.21.15.96:0/2277448706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb06ffff640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 0x7fb05c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 0x55887852f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852f730 0x558878531bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb06d7fa640 1 -- 172.21.15.96:0/2234453695 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb0680cf150 con 0x7fb0701033b0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 -- 172.21.15.96:0/2277448706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852f730 msgr2=0x558878531bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852f730 0x558878531bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 -- 172.21.15.96:0/2277448706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 msgr2=0x55887852f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 0x55887852f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb066ffd640 1 -- 172.21.15.96:0/2277448706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558878458ff0 con 0x558878529bd0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.323+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 0x55887852f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb06ffff640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 0x7fb05c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb068097f40 tx=0x7fb0680668a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb058013840 tx=0x7fb058005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39531ad618c2855 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb065ffb640 1 -- 172.21.15.96:0/2277448706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058013170 con 0x558878529bd0 2024-03-18T15:39:57.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb065ffb640 1 -- 172.21.15.96:0/2277448706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb058005500 con 0x558878529bd0 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb065ffb640 1 -- 172.21.15.96:0/2277448706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058004d50 con 0x558878529bd0 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 -- 172.21.15.96:0/2277448706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 msgr2=0x558878529ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb058013840 tx=0x7fb058005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 -- 172.21.15.96:0/2277448706 shutdown_connections 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852f730 0x558878531bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852a5c0 0x55887852f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 --2- 172.21.15.96:0/2277448706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558878529bd0 0x558878529ff0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 -- 172.21.15.96:0/2277448706 >> 172.21.15.96:0/2277448706 conn(0x558878525140 msgr2=0x558878525540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 -- 172.21.15.96:0/2277448706 shutdown_connections 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.324+0000 7fb07878c740 1 -- 172.21.15.96:0/2277448706 wait complete. 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 -- start start 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 Processor -- start 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x558878533ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x5588785341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558878532680 con 0x55887852f730 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558878449ef0 con 0x558878529bd0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558878532800 con 0x55887852a5c0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb07878c740 10 client.0 _ll_register_callbacks cb 0x558878444dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53856/0 (socket says 172.21.15.96:53856) 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb066ffd640 1 -- 172.21.15.96:0/1650514835 learned_addr learned my addr 172.21.15.96:0/1650514835 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x5588785341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.325+0000 7fb066ffd640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x558878533ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 -- 172.21.15.96:0/1650514835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 msgr2=0x5588785341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x5588785341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 -- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 msgr2=0x558878533ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x558878533ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 -- 172.21.15.96:0/1650514835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558878456a30 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x5588785341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb066ffd640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb0580046f0 tx=0x7fb058009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a66d7ae2249ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058004ad0 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb058004c70 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb058005500 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb058004e70 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb07878c740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588781a22d0 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb058005500 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb05800fc80 con 0x55887852f730 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.326+0000 7fb07878c740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb07878c740 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb07878c740 20 client.15264 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558878540440 0x5588785418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb07878c740 4 client.15264 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55887849b1c0 con 0x558878540440 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558878540440 0x5588785418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558878540440 0x5588785418b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3feaca6e1f75d44e server_cookie=d9f057d77c701ad5 in_seq=0 out_seq=0 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.327+0000 7fb04effd640 10 client.15264 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb04effd640 1 -- 172.21.15.96:0/1650514835 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55887849b1c0 con 0x558878540440 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb04effd640 10 client.15264 handle_command_reply: tid=0 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 client.15264 shutdown 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 msgr2=0x55887853b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb0580046f0 tx=0x7fb058009260 comp rx=0 tx=0).stop 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 shutdown_connections 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558878540440 0x5588785418b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x55887853b220 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x5588785341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 --2- 172.21.15.96:0/1650514835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x558878533ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb04effd640 0 client.15264 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 >> 172.21.15.96:0/1650514835 conn(0x558878525140 msgr2=0x558878524380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 shutdown_connections 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 1 -- 172.21.15.96:0/1650514835 wait complete. 2024-03-18T15:39:57.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.328+0000 7fb07878c740 20 client.15264 trim_cache size 0 max 16384 2024-03-18T15:39:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 Processor -- start 2024-03-18T15:39:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 -- start start 2024-03-18T15:39:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558878529bd0 0x558878534140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852f730 0x558878523af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55887853fbf0 con 0x55887852a5c0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5588785402c0 con 0x55887852f730 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb07878c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558878540990 con 0x558878529bd0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53862/0 (socket says 172.21.15.96:53862) 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 -- 172.21.15.96:0/283046688 learned_addr learned my addr 172.21.15.96:0/283046688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb066ffd640 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558878529bd0 0x558878534140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb066ffd640 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852f730 0x558878523af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 -- 172.21.15.96:0/283046688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558878529bd0 msgr2=0x558878534140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558878529bd0 0x558878534140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 -- 172.21.15.96:0/283046688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852f730 msgr2=0x558878523af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852f730 0x558878523af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 -- 172.21.15.96:0/283046688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887826c6b0 con 0x55887852a5c0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.346+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb06002c7b0 tx=0x7fb06002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6ce2a50374538e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb0657fa640 1 -- 172.21.15.96:0/283046688 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06004b070 con 0x55887852a5c0 2024-03-18T15:39:57.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb0657fa640 1 -- 172.21.15.96:0/283046688 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb060004380 con 0x55887852a5c0 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb0657fa640 1 -- 172.21.15.96:0/283046688 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060005400 con 0x55887852a5c0 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- 172.21.15.96:0/283046688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 msgr2=0x558878531b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb06002c7b0 tx=0x7fb06002e040 comp rx=0 tx=0).stop 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- 172.21.15.96:0/283046688 shutdown_connections 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55887852f730 0x558878523af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852a5c0 0x558878531b20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 --2- 172.21.15.96:0/283046688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558878529bd0 0x558878534140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- 172.21.15.96:0/283046688 >> 172.21.15.96:0/283046688 conn(0x558878525140 msgr2=0x55887853f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- 172.21.15.96:0/283046688 shutdown_connections 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- 172.21.15.96:0/283046688 wait complete. 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.347+0000 7fb07878c740 1 -- start start 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 Processor -- start 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x55887852d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x55887852dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558878541730 con 0x55887852f730 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558878541060 con 0x558878529bd0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558878541e00 con 0x55887852a5c0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb07878c740 10 client.0 _ll_register_callbacks cb 0x7fb0680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53872/0 (socket says 172.21.15.96:53872) 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 -- 172.21.15.96:0/2371789249 learned_addr learned my addr 172.21.15.96:0/2371789249 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x55887852dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x55887852d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 -- 172.21.15.96:0/2371789249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 msgr2=0x55887852dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x55887852dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 -- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 msgr2=0x55887852d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x55887852d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 -- 172.21.15.96:0/2371789249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588785a8ce0 con 0x55887852f730 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb066ffd640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb058013840 tx=0x7fb058015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5733c85ab760dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.348+0000 7fb04effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb058004930 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7fb058005490 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060004380 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb058005690 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 10 client.15270 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588785a9fe0 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb060004380 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb058003eb0 con 0x55887852f730 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 10 client.15270 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 20 client.15270 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5588785aeab0 0x5588785ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 4 client.15270 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "status"}) -- 0x558878495510 con 0x5588785aeab0 2024-03-18T15:39:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.349+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5588785aeab0 0x5588785ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb06f7fe640 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5588785aeab0 0x5588785ad7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee9c72f488101f11 server_cookie=2015abc35d291393 in_seq=0 out_seq=0 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb04effd640 10 client.15270 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb04effd640 1 -- 172.21.15.96:0/2371789249 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558878495510 con 0x5588785aeab0 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb04effd640 10 client.15270 handle_command_reply: tid=0 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 client.15270 shutdown 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 msgr2=0x5588785a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb058013840 tx=0x7fb058015040 comp rx=0 tx=0).stop 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 shutdown_connections 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5588785aeab0 0x5588785ad7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55887852f730 0x5588785a86d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55887852a5c0 0x55887852dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb04effd640 0 client.15270 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 --2- 172.21.15.96:0/2371789249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558878529bd0 0x55887852d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 >> 172.21.15.96:0/2371789249 conn(0x558878525140 msgr2=0x558878531b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 shutdown_connections 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 1 -- 172.21.15.96:0/2371789249 wait complete. 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.350+0000 7fb07878c740 20 client.15270 trim_cache size 0 max 16384 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 4, 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24452, 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 107.159748329, 2024-03-18T15:39:57.358 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:57.359 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 130.79301146700001 2024-03-18T15:39:57.359 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:57.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 msgr2=0x7fb05c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 0x7fb05c0a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb068097f40 tx=0x7fb0680668a0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 msgr2=0x7fb07019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb06800eb10 tx=0x7fb06800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 shutdown_connections 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb05c0a2800 0x7fb05c0a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0701033b0 0x7fb07019f0c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0701029f0 0x7fb070198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 --2- 172.21.15.96:0/2234453695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb070101800 0x7fb070197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 >> 172.21.15.96:0/2234453695 conn(0x7fb0700fcd50 msgr2=0x7fb070105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 shutdown_connections 2024-03-18T15:39:57.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.351+0000 7fb075ac6640 1 -- 172.21.15.96:0/2234453695 wait complete. 2024-03-18T15:39:57.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 -- 172.21.15.96:0/2053093136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81006f0 msgr2=0x7f25b8100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 --2- 172.21.15.96:0/2053093136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81006f0 0x7f25b8100b10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009900 tx=0x7f25ac02bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 -- 172.21.15.96:0/2053093136 shutdown_connections 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 --2- 172.21.15.96:0/2053093136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 0x7f25b8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 --2- 172.21.15.96:0/2053093136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81018e0 0x7f25b8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 --2- 172.21.15.96:0/2053093136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81006f0 0x7f25b8100b10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 -- 172.21.15.96:0/2053093136 >> 172.21.15.96:0/2053093136 conn(0x7f25b80fbc80 msgr2=0x7f25b80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 -- 172.21.15.96:0/2053093136 shutdown_connections 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.463+0000 7f25beff3640 1 -- 172.21.15.96:0/2053093136 wait complete. 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 Processor -- start 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 -- start start 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81006f0 0x7f25b806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 0x7f25b819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810c340 con 0x7f25b81018e0 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810c1c0 con 0x7f25b81022a0 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25b810c4c0 con 0x7f25b81006f0 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53888/0 (socket says 172.21.15.96:53888) 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 -- 172.21.15.96:0/2650969611 learned_addr learned my addr 172.21.15.96:0/2650969611 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25bd590640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 0x7f25b819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81006f0 0x7f25b806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 -- 172.21.15.96:0/2650969611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81006f0 msgr2=0x7f25b806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81006f0 0x7f25b806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 -- 172.21.15.96:0/2650969611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 msgr2=0x7f25b819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 0x7f25b819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 -- 172.21.15.96:0/2650969611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b819f7f0 con 0x7f25b81018e0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b7fff640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9f0 tx=0x7f25a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b5ffb640 1 -- 172.21.15.96:0/2650969611 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a400ccc0 con 0x7f25b81018e0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b5ffb640 1 -- 172.21.15.96:0/2650969611 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a4010910 con 0x7f25b81018e0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b819fa80 con 0x7f25b81018e0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.464+0000 7f25b5ffb640 1 -- 172.21.15.96:0/2650969611 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a40188a0 con 0x7f25b81018e0 2024-03-18T15:39:57.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.465+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b819ffd0 con 0x7f25b81018e0 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b5ffb640 1 -- 172.21.15.96:0/2650969611 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f25a4010ac0 con 0x7f25b81018e0 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 Processor -- start 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 -- start start 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b5ffb640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 0x7f25a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 0x559f542b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f54214c80 con 0x559f542b6730 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f541bdaa0 con 0x559f542b15c0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f54210d20 con 0x559f542b0bd0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25bd590640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 0x7f25a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25bd590640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25bcd8f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 0x559f542b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:36372/0 (socket says 172.21.15.96:36372) 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b7fff640 1 -- 172.21.15.96:0/3785092426 learned_addr learned my addr 172.21.15.96:0/3785092426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.467+0000 7f25b5ffb640 1 -- 172.21.15.96:0/2650969611 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f25a40ceeb0 con 0x7f25b81018e0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 -- 172.21.15.96:0/3785092426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 msgr2=0x559f542b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 0x559f542b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 -- 172.21.15.96:0/3785092426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 msgr2=0x559f542b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 -- 172.21.15.96:0/3785092426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f541dfff0 con 0x559f542b15c0 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25bd590640 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 0x559f542b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25bd590640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 0x7f25a80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f25b0005fd0 tx=0x7f25b0009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25b7fff640 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a4097ca0 tx=0x7f25a4009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4315bf538532540c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25a2ffd640 1 -- 172.21.15.96:0/3785092426 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a40cce60 con 0x559f542b15c0 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25a2ffd640 1 -- 172.21.15.96:0/3785092426 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a409bf40 con 0x559f542b15c0 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.468+0000 7f25a2ffd640 1 -- 172.21.15.96:0/3785092426 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4026040 con 0x559f542b15c0 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- 172.21.15.96:0/3785092426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 msgr2=0x559f542b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a4097ca0 tx=0x7f25a4009700 comp rx=0 tx=0).stop 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- 172.21.15.96:0/3785092426 shutdown_connections 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b15c0 0x559f542b60c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/3785092426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b0bd0 0x559f542b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- 172.21.15.96:0/3785092426 >> 172.21.15.96:0/3785092426 conn(0x559f542ac140 msgr2=0x559f542ac540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- 172.21.15.96:0/3785092426 shutdown_connections 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- 172.21.15.96:0/3785092426 wait complete. 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 -- start start 2024-03-18T15:39:57.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 Processor -- start 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542bb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.469+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f542b9620 con 0x559f542b6730 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f541d0ef0 con 0x559f542b0bd0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f542b97a0 con 0x559f542b15c0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 10 client.0 _ll_register_callbacks cb 0x559f541cbdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bcd8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53906/0 (socket says 172.21.15.96:53906) 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 -- 172.21.15.96:0/2235245703 learned_addr learned my addr 172.21.15.96:0/2235245703 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25b7fff640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542bb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 -- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 msgr2=0x559f542bb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542bb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 -- 172.21.15.96:0/2235245703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 msgr2=0x559f542bacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 -- 172.21.15.96:0/2235245703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f541dda30 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25bd590640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25b0011fd0 tx=0x7f25b0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f67597176bf37b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f2597fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0024070 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a409bf40 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f25b0004740 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25b0029460 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 10 client.15282 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.470+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559f53f292d0 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25a409bf40 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f25b0034650 con 0x559f542b6730 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 10 client.15282 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 20 client.15282 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f542c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 4 client.15282 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f542221c0 con 0x559f542c77f0 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f542c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f2597fff640 10 client.15282 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.471+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f542c8c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9dea1b3f719fcc61 server_cookie=6efdfdbf0e45d04e in_seq=0 out_seq=0 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f2597fff640 1 -- 172.21.15.96:0/2235245703 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559f542221c0 con 0x559f542c77f0 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f2597fff640 10 client.15282 handle_command_reply: tid=0 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 client.15282 shutdown 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 msgr2=0x559f542c2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25b0011fd0 tx=0x7f25b0009500 comp rx=0 tx=0).stop 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 shutdown_connections 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f542c8c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b6730 0x559f542c2240 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542bb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f2597fff640 0 client.15282 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/2235245703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 >> 172.21.15.96:0/2235245703 conn(0x559f542ac140 msgr2=0x559f542ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 shutdown_connections 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 1 -- 172.21.15.96:0/2235245703 wait complete. 2024-03-18T15:39:57.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.472+0000 7f25c1bdb740 20 client.15282 trim_cache size 0 max 16384 2024-03-18T15:39:57.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 Processor -- start 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 -- start start 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b6730 0x559f542bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f542c6fa0 con 0x559f542b0bd0 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f542c7670 con 0x559f542b6730 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f542c7d40 con 0x559f542b15c0 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25bcd8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25bcd8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53916/0 (socket says 172.21.15.96:53916) 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.490+0000 7f25bcd8f640 1 -- 172.21.15.96:0/47038566 learned_addr learned my addr 172.21.15.96:0/47038566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25b7fff640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bd590640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b6730 0x559f542bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 -- 172.21.15.96:0/47038566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 msgr2=0x559f542c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 -- 172.21.15.96:0/47038566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b6730 msgr2=0x559f542bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b6730 0x559f542bae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 -- 172.21.15.96:0/47038566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f53ff36b0 con 0x559f542b0bd0 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25b7fff640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02cfd0 tx=0x7f25ac02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9442052d23bde9a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25a27fc640 1 -- 172.21.15.96:0/47038566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac046ca0 con 0x559f542b0bd0 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25a27fc640 1 -- 172.21.15.96:0/47038566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25ac04b070 con 0x559f542b0bd0 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25a27fc640 1 -- 172.21.15.96:0/47038566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac005040 con 0x559f542b0bd0 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25c1bdb740 1 -- 172.21.15.96:0/47038566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 msgr2=0x559f542ba8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f25ac02cfd0 tx=0x7f25ac02e040 comp rx=0 tx=0).stop 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25c1bdb740 1 -- 172.21.15.96:0/47038566 shutdown_connections 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b6730 0x559f542bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.491+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b15c0 0x559f542c2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/47038566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b0bd0 0x559f542ba8d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- 172.21.15.96:0/47038566 >> 172.21.15.96:0/47038566 conn(0x559f542ac140 msgr2=0x559f542c65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- 172.21.15.96:0/47038566 shutdown_connections 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- 172.21.15.96:0/47038566 wait complete. 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- start start 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 Processor -- start 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b6730 0x559f542c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559f542c8ae0 con 0x559f542b15c0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559f542c8410 con 0x559f542b0bd0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f542c91b0 con 0x559f542b6730 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25c1bdb740 10 client.0 _ll_register_callbacks cb 0x7f25b00116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53924/0 (socket says 172.21.15.96:53924) 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25b7fff640 1 -- 172.21.15.96:0/658873091 learned_addr learned my addr 172.21.15.96:0/658873091 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25bcd8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.492+0000 7f25bd590640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b6730 0x559f542c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 -- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b6730 msgr2=0x559f542c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b6730 0x559f542c08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 -- 172.21.15.96:0/658873091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 msgr2=0x559f542bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 -- 172.21.15.96:0/658873091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f542c0e90 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25b7fff640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ccd10 tx=0x7f25a40b4be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918c5677e2d2c787 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a40cce60 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f25a40860b0 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25ac04b070 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f25a40b3040 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 10 client.15288 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559f542c2190 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25ac04b070 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f25a409cb70 con 0x559f542b15c0 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 10 client.15288 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 10 client.15288 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 20 client.15288 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f54330330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 4 client.15288 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.493+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "status"}) -- 0x559f5421c510 con 0x559f542c77f0 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f54330330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f2597fff640 10 client.15288 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25bcd8f640 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f54330330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=915abfc3745f5d0 server_cookie=44fa5e570ea71780 in_seq=0 out_seq=0 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f2597fff640 1 -- 172.21.15.96:0/658873091 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x559f5421c510 con 0x559f542c77f0 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f2597fff640 10 client.15288 handle_command_reply: tid=0 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 client.15288 shutdown 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 msgr2=0x559f542bd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f25a40ccd10 tx=0x7f25a40b4be0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 shutdown_connections 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559f542c77f0 0x559f54330330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f542b6730 0x559f542c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559f542b15c0 0x559f542bd880 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f2597fff640 0 client.15288 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 --2- 172.21.15.96:0/658873091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f542b0bd0 0x559f542bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 >> 172.21.15.96:0/658873091 conn(0x559f542ac140 msgr2=0x559f542ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 shutdown_connections 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 1 -- 172.21.15.96:0/658873091 wait complete. 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.494+0000 7f25c1bdb740 20 client.15288 trim_cache size 0 max 16384 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 3, 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24460, 2024-03-18T15:39:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 108.32387537, 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 136.588276597 2024-03-18T15:39:57.503 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 msgr2=0x7f25a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 0x7f25a80a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f25b0005fd0 tx=0x7f25b0009f30 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 msgr2=0x7f25b819c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9f0 tx=0x7f25a400edf0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 shutdown_connections 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25a80a2800 0x7f25a80a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25b81022a0 0x7f25b819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b81018e0 0x7f25b819c8c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 --2- 172.21.15.96:0/2650969611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b81006f0 0x7f25b806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 >> 172.21.15.96:0/2650969611 conn(0x7f25b80fbc80 msgr2=0x7f25b8104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 shutdown_connections 2024-03-18T15:39:57.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.495+0000 7f25beff3640 1 -- 172.21.15.96:0/2650969611 wait complete. 2024-03-18T15:39:57.507 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 -- 172.21.15.96:0/3824184553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001006f0 msgr2=0x7f7300100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 --2- 172.21.15.96:0/3824184553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001006f0 0x7f7300100b10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009ca0 tx=0x7f72fc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 -- 172.21.15.96:0/3824184553 shutdown_connections 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 --2- 172.21.15.96:0/3824184553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 0x7f7300108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 --2- 172.21.15.96:0/3824184553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001018e0 0x7f7300101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 --2- 172.21.15.96:0/3824184553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001006f0 0x7f7300100b10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 -- 172.21.15.96:0/3824184553 >> 172.21.15.96:0/3824184553 conn(0x7f73000fbc80 msgr2=0x7f73000fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 -- 172.21.15.96:0/3824184553 shutdown_connections 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.608+0000 7f7307e61640 1 -- 172.21.15.96:0/3824184553 wait complete. 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 Processor -- start 2024-03-18T15:39:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- start start 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001006f0 0x7f730006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 0x7f730019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f730010c370 con 0x7f73001018e0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f730010c1f0 con 0x7f73001006f0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f730010c4f0 con 0x7f73001022a0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53928/0 (socket says 172.21.15.96:53928) 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 -- 172.21.15.96:0/2176682704 learned_addr learned my addr 172.21.15.96:0/2176682704 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73063fe640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 0x7f730019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7305bfd640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001006f0 0x7f730006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 -- 172.21.15.96:0/2176682704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 msgr2=0x7f730019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 0x7f730019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 -- 172.21.15.96:0/2176682704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001006f0 msgr2=0x7f730006a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001006f0 0x7f730006a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 -- 172.21.15.96:0/2176682704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730019f7d0 con 0x7f73001018e0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f73053fc640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d950 tx=0x7f72f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f72f6ffd640 1 -- 172.21.15.96:0/2176682704 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0017030 con 0x7f73001018e0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f72f6ffd640 1 -- 172.21.15.96:0/2176682704 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f0004020 con 0x7f73001018e0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730019fa60 con 0x7f73001018e0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f72f6ffd640 1 -- 172.21.15.96:0/2176682704 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f00047a0 con 0x7f73001018e0 2024-03-18T15:39:57.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.609+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730019ffb0 con 0x7f73001018e0 2024-03-18T15:39:57.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.612+0000 7f72f6ffd640 1 -- 172.21.15.96:0/2176682704 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f72f00041d0 con 0x7f73001018e0 2024-03-18T15:39:57.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.612+0000 7f730aa55740 1 Processor -- start 2024-03-18T15:39:57.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.612+0000 7f730aa55740 1 -- start start 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.612+0000 7f730aa55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.612+0000 7f730aa55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 0x5631cd5b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f730aa55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b5730 0x5631cd5b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f730aa55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cd513c80 con 0x5631cd5b5730 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f730aa55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cd4bcaa0 con 0x5631cd5b05c0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f730aa55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cd50fd20 con 0x5631cd5afbd0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f72f6ffd640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 0x7f72ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b5730 0x5631cd5b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:46494/0 (socket says 172.21.15.96:46494) 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 0x5631cd5b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 -- 172.21.15.96:0/660331527 learned_addr learned my addr 172.21.15.96:0/660331527 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 0x7f72ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f72f6ffd640 1 -- 172.21.15.96:0/2176682704 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f72f00ce440 con 0x7f73001018e0 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 -- 172.21.15.96:0/660331527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 msgr2=0x5631cd5afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 -- 172.21.15.96:0/660331527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b5730 msgr2=0x5631cd5b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b5730 0x5631cd5b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 -- 172.21.15.96:0/660331527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cd4deff0 con 0x5631cd5b05c0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f73063fe640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 0x7f72ec0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5631cd5b0e20 tx=0x7f72f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f7305bfd640 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 0x5631cd5b50c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f72fc02c630 tx=0x7f72fc0323d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2638feee36ee254d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f72ebfff640 1 -- 172.21.15.96:0/660331527 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc043070 con 0x5631cd5b05c0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f72ebfff640 1 -- 172.21.15.96:0/660331527 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72fc02cca0 con 0x5631cd5b05c0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.613+0000 7f72ebfff640 1 -- 172.21.15.96:0/660331527 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc039040 con 0x5631cd5b05c0 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- 172.21.15.96:0/660331527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 msgr2=0x5631cd5b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 0x5631cd5b50c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f72fc02c630 tx=0x7f72fc0323d0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- 172.21.15.96:0/660331527 shutdown_connections 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b5730 0x5631cd5b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b05c0 0x5631cd5b50c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 --2- 172.21.15.96:0/660331527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- 172.21.15.96:0/660331527 >> 172.21.15.96:0/660331527 conn(0x5631cd5ab140 msgr2=0x5631cd5ab540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- 172.21.15.96:0/660331527 shutdown_connections 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- 172.21.15.96:0/660331527 wait complete. 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.614+0000 7f730aa55740 1 -- start start 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 Processor -- start 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b05c0 0x5631cd5ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5b8680 con 0x5631cd5afbd0 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cd4cfef0 con 0x5631cd5b5730 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5b8800 con 0x5631cd5b05c0 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f730aa55740 10 client.0 _ll_register_callbacks cb 0x5631cd4cadc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53946/0 (socket says 172.21.15.96:53946) 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 -- 172.21.15.96:0/2899993613 learned_addr learned my addr 172.21.15.96:0/2899993613 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f73053fc640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b05c0 0x5631cd5ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f73063fe640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 -- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b05c0 msgr2=0x5631cd5ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b05c0 0x5631cd5ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 -- 172.21.15.96:0/2899993613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 msgr2=0x5631cd5c1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5c1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 -- 172.21.15.96:0/2899993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cd4dca30 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f7305bfd640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005e30 tx=0x7f72fc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f66131baf628ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.615+0000 7f72e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc04c030 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f72fc0328a0 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72fc02cca0 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72fc039070 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 10 client.15300 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5631cd2282d0 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72fc02cca0 con 0x5631cd5afbd0 2024-03-18T15:39:57.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f72fc03f600 con 0x5631cd5afbd0 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 10 client.15300 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 10 client.15300 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 20 client.15300 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd5c6280 0x5631cd5c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 4 client.15300 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.616+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631cd5211c0 con 0x5631cd5c6280 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.617+0000 7f73053fc640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd5c6280 0x5631cd5c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.617+0000 7f72e97fa640 10 client.15300 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.617+0000 7f73053fc640 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd5c6280 0x5631cd5c76f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=51a00f39000febcd server_cookie=5dd5f9c49eab68cc in_seq=0 out_seq=0 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f72e97fa640 1 -- 172.21.15.96:0/2899993613 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5631cd5211c0 con 0x5631cd5c6280 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f72e97fa640 10 client.15300 handle_command_reply: tid=0 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 client.15300 shutdown 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 msgr2=0x5631cd5b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005e30 tx=0x7f72fc003970 comp rx=0 tx=0).stop 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 shutdown_connections 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd5c6280 0x5631cd5c76f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5c1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f72e97fa640 0 client.15300 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b05c0 0x5631cd5ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 --2- 172.21.15.96:0/2899993613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5afbd0 0x5631cd5b9de0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 >> 172.21.15.96:0/2899993613 conn(0x5631cd5ab140 msgr2=0x5631cd5aa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 shutdown_connections 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 1 -- 172.21.15.96:0/2899993613 wait complete. 2024-03-18T15:39:57.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.618+0000 7f730aa55740 20 client.15300 trim_cache size 0 max 16384 2024-03-18T15:39:57.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 Processor -- start 2024-03-18T15:39:57.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 -- start start 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c5a30 con 0x5631cd5b05c0 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c6100 con 0x5631cd5b5730 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f730aa55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c67d0 con 0x5631cd5afbd0 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53948/0 (socket says 172.21.15.96:53948) 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 -- 172.21.15.96:0/1628544826 learned_addr learned my addr 172.21.15.96:0/1628544826 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f7305bfd640 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73063fe640 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 -- 172.21.15.96:0/1628544826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 msgr2=0x5631cd5ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 -- 172.21.15.96:0/1628544826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 msgr2=0x5631cd5b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 -- 172.21.15.96:0/1628544826 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cd2f26b0 con 0x5631cd5b05c0 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.636+0000 7f73053fc640 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f72f0004770 tx=0x7f72f006db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b345866c65e525f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f72eb7fe640 1 -- 172.21.15.96:0/1628544826 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f009ef20 con 0x5631cd5b05c0 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f72eb7fe640 1 -- 172.21.15.96:0/1628544826 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f00a3570 con 0x5631cd5b05c0 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f72eb7fe640 1 -- 172.21.15.96:0/1628544826 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f009a220 con 0x5631cd5b05c0 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 -- 172.21.15.96:0/1628544826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 msgr2=0x5631cd5c1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f72f0004770 tx=0x7f72f006db00 comp rx=0 tx=0).stop 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 -- 172.21.15.96:0/1628544826 shutdown_connections 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5b5730 0x5631cd5b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5c1320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 --2- 172.21.15.96:0/1628544826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5afbd0 0x5631cd5ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 -- 172.21.15.96:0/1628544826 >> 172.21.15.96:0/1628544826 conn(0x5631cd5ab140 msgr2=0x5631cd5aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 -- 172.21.15.96:0/1628544826 shutdown_connections 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.637+0000 7f730aa55740 1 -- 172.21.15.96:0/1628544826 wait complete. 2024-03-18T15:39:57.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 -- start start 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 Processor -- start 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 0x5631cd5bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b5730 0x5631cd5bfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c7570 con 0x5631cd5b05c0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c6ea0 con 0x5631cd5afbd0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cd5c7c40 con 0x5631cd5b5730 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f730aa55740 10 client.0 _ll_register_callbacks cb 0x7f72f00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f7305bfd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 0x5631cd5bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f73053fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53964/0 (socket says 172.21.15.96:53964) 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f73053fc640 1 -- 172.21.15.96:0/1437841398 learned_addr learned my addr 172.21.15.96:0/1437841398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.638+0000 7f73063fe640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b5730 0x5631cd5bfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 -- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b5730 msgr2=0x5631cd5bfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b5730 0x5631cd5bfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 -- 172.21.15.96:0/1437841398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 msgr2=0x5631cd5bc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 0x5631cd5bc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 -- 172.21.15.96:0/1437841398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cd5c0350 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f7305bfd640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 0x5631cd5bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f73053fc640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009610 tx=0x7f72f009acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b56cfc0b5efa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f009ef20 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f72f00ce150 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72f00a3570 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72f009a220 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 10 client.15306 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5631cd5c1650 con 0x5631cd5b05c0 2024-03-18T15:39:57.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f00a3570 con 0x5631cd5b05c0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f72f009fe40 con 0x5631cd5b05c0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 10 client.15306 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 10 client.15306 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 20 client.15306 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd6302e0 0x5631cd62cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 4 client.15306 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.639+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "status"}) -- 0x5631cd51b510 con 0x5631cd6302e0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f7305bfd640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd6302e0 0x5631cd62cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f72e97fa640 10 client.15306 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f7305bfd640 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd6302e0 0x5631cd62cdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5cad91319bae447 server_cookie=825a8fa205afbc15 in_seq=0 out_seq=0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f72e97fa640 1 -- 172.21.15.96:0/1437841398 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5631cd51b510 con 0x5631cd6302e0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f72e97fa640 10 client.15306 handle_command_reply: tid=0 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 client.15306 shutdown 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 msgr2=0x5631cd5bcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009610 tx=0x7f72f009acc0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 shutdown_connections 2024-03-18T15:39:57.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5631cd6302e0 0x5631cd62cdf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cd5b5730 0x5631cd5bfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cd5b05c0 0x5631cd5bcd90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 --2- 172.21.15.96:0/1437841398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cd5afbd0 0x5631cd5bc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f72e97fa640 0 client.15306 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 >> 172.21.15.96:0/1437841398 conn(0x5631cd5ab140 msgr2=0x5631cd5aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 shutdown_connections 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 1 -- 172.21.15.96:0/1437841398 wait complete. 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.640+0000 7f730aa55740 20 client.15306 trim_cache size 0 max 16384 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:57.648 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24472, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 109.475104446, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 133.99549673000001 2024-03-18T15:39:57.649 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 msgr2=0x7f72ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 0x7f72ec0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5631cd5b0e20 tx=0x7f72f8005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 msgr2=0x7f730019c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d950 tx=0x7f72f000de10 comp rx=0 tx=0).stop 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 shutdown_connections 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f72ec0a2800 0x7f72ec0a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73001022a0 0x7f730019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73001018e0 0x7f730019c8a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 --2- 172.21.15.96:0/2176682704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73001006f0 0x7f730006a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 >> 172.21.15.96:0/2176682704 conn(0x7f73000fbc80 msgr2=0x7f7300104c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 shutdown_connections 2024-03-18T15:39:57.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.641+0000 7f7307e61640 1 -- 172.21.15.96:0/2176682704 wait complete. 2024-03-18T15:39:57.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.754+0000 7f252c35b640 1 -- 172.21.15.96:0/4043006104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 msgr2=0x7f25241082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.754+0000 7f252c35b640 1 --2- 172.21.15.96:0/4043006104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f25241082c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f251c009ca0 tx=0x7f251c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- 172.21.15.96:0/4043006104 shutdown_connections 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- 172.21.15.96:0/4043006104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f25241082c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- 172.21.15.96:0/4043006104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 0x7f2524101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- 172.21.15.96:0/4043006104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 0x7f25241002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- 172.21.15.96:0/4043006104 >> 172.21.15.96:0/4043006104 conn(0x7f25240fb3e0 msgr2=0x7f25240fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- 172.21.15.96:0/4043006104 shutdown_connections 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- 172.21.15.96:0/4043006104 wait complete. 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 Processor -- start 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- start start 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 0x7f2524197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 0x7f2524197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f252410b2c0 con 0x7f25241019a0 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f252410b140 con 0x7f25240ffe90 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252c35b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f252410b440 con 0x7f2524100fe0 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252a8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252a8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53984/0 (socket says 172.21.15.96:53984) 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252a0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 0x7f2524197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.755+0000 7f252a8f8640 1 -- 172.21.15.96:0/1679637477 learned_addr learned my addr 172.21.15.96:0/1679637477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f25298f6640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 0x7f2524197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 -- 172.21.15.96:0/1679637477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 msgr2=0x7f2524197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 0x7f2524197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 -- 172.21.15.96:0/1679637477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 msgr2=0x7f2524197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 0x7f2524197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 -- 172.21.15.96:0/1679637477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252419f6d0 con 0x7f25241019a0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252a8f8640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f251c037040 tx=0x7f251c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f251b7fe640 1 -- 172.21.15.96:0/1679637477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c041020 con 0x7f25241019a0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f251b7fe640 1 -- 172.21.15.96:0/1679637477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c034cd0 con 0x7f25241019a0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252419f960 con 0x7f25241019a0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f251b7fe640 1 -- 172.21.15.96:0/1679637477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c039040 con 0x7f25241019a0 2024-03-18T15:39:57.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.756+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252419fe50 con 0x7f25241019a0 2024-03-18T15:39:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.758+0000 7f251b7fe640 1 -- 172.21.15.96:0/1679637477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f251c044020 con 0x7f25241019a0 2024-03-18T15:39:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 Processor -- start 2024-03-18T15:39:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 -- start start 2024-03-18T15:39:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f251b7fe640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b289d0 0x55fad1b2d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 0x55fad1b2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad1a87c80 con 0x55fad1b27fe0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fad1a30aa0 con 0x55fad1b2db40 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252f029740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1a83d20 con 0x55fad1b289d0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f25298f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f252a8f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 0x55fad1b2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f25298f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53996/0 (socket says 172.21.15.96:53996) 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f25298f6640 1 -- 172.21.15.96:0/4274035315 learned_addr learned my addr 172.21.15.96:0/4274035315 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f25298f6640 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b289d0 0x55fad1b2d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.759+0000 7f251b7fe640 1 -- 172.21.15.96:0/1679637477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f251c0f0910 con 0x7f25241019a0 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 -- 172.21.15.96:0/4274035315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b289d0 msgr2=0x55fad1b2d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b289d0 0x55fad1b2d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 -- 172.21.15.96:0/4274035315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 msgr2=0x55fad1b28400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 -- 172.21.15.96:0/4274035315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad1a52ff0 con 0x55fad1b2db40 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f25298f6640 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 0x7f25100a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25140095f0 tx=0x7f2514005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f252a8f8640 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 0x55fad1b2ffd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f251c0ee450 tx=0x7f251c02c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690c5754ae432fec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f2518bfe640 1 -- 172.21.15.96:0/4274035315 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0bce70 con 0x55fad1b2db40 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f2518bfe640 1 -- 172.21.15.96:0/4274035315 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c0bda30 con 0x55fad1b2db40 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.760+0000 7f2518bfe640 1 -- 172.21.15.96:0/4274035315 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c0bcc90 con 0x55fad1b2db40 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- 172.21.15.96:0/4274035315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 msgr2=0x55fad1b2ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 0x55fad1b2ffd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f251c0ee450 tx=0x7f251c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- 172.21.15.96:0/4274035315 shutdown_connections 2024-03-18T15:39:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b2db40 0x55fad1b2ffd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b289d0 0x55fad1b2d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- 172.21.15.96:0/4274035315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b28400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- 172.21.15.96:0/4274035315 >> 172.21.15.96:0/4274035315 conn(0x55fad1b1f140 msgr2=0x55fad1b1f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- 172.21.15.96:0/4274035315 shutdown_connections 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- 172.21.15.96:0/4274035315 wait complete. 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- start start 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 Processor -- start 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b308b0 con 0x55fad1b27fe0 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fad1a43ef0 con 0x55fad1b289d0 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b30a30 con 0x55fad1b2db40 2024-03-18T15:39:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252a0f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252f029740 10 client.0 _ll_register_callbacks cb 0x55fad1a3edc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252a0f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54008/0 (socket says 172.21.15.96:54008) 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.761+0000 7f252a0f7640 1 -- 172.21.15.96:0/469617649 learned_addr learned my addr 172.21.15.96:0/469617649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f25298f6640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a8f8640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 -- 172.21.15.96:0/469617649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 msgr2=0x55fad1b39410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b39410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 -- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 msgr2=0x55fad1b323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 -- 172.21.15.96:0/469617649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad1a50a30 con 0x55fad1b27fe0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252a0f7640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2514003970 tx=0x7f2514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc05cb0a9096c45c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251400ecc0 con 0x55fad1b27fe0 2024-03-18T15:39:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f251401e420 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251c0bda30 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f251400ee60 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252f029740 10 client.15318 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fad179c2d0 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251c0bda30 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.762+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f2514031740 con 0x55fad1b27fe0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 10 client.15318 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 20 client.15318 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1b3e9d0 0x55fad1b3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 4 client.15318 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad1a951c0 con 0x55fad1b3e9d0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f25298f6640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1b3e9d0 0x55fad1b3fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f250dffb640 10 client.15318 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.763+0000 7f25298f6640 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1b3e9d0 0x55fad1b3fe40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=957836dfb8d9308 server_cookie=f90d6ad05b8d635f in_seq=0 out_seq=0 2024-03-18T15:39:57.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f250dffb640 1 -- 172.21.15.96:0/469617649 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fad1a951c0 con 0x55fad1b3e9d0 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f250dffb640 10 client.15318 handle_command_reply: tid=0 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 client.15318 shutdown 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 msgr2=0x55fad1b31e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2514003970 tx=0x7f2514004730 comp rx=0 tx=0).stop 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 shutdown_connections 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b39410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1b3e9d0 0x55fad1b3fe40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 --2- 172.21.15.96:0/469617649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b31e90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f250dffb640 0 client.15318 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 >> 172.21.15.96:0/469617649 conn(0x55fad1b1f140 msgr2=0x55fad1b1e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 shutdown_connections 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 1 -- 172.21.15.96:0/469617649 wait complete. 2024-03-18T15:39:57.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.764+0000 7f252f029740 20 client.15318 trim_cache size 0 max 16384 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 Processor -- start 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 -- start start 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b27fe0 0x55fad1b3d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b31aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b3e180 con 0x55fad1b289d0 2024-03-18T15:39:57.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b3e850 con 0x55fad1b27fe0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252f029740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b3ef20 con 0x55fad1b2db40 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54012/0 (socket says 172.21.15.96:54012) 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 -- 172.21.15.96:0/1227831664 learned_addr learned my addr 172.21.15.96:0/1227831664 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b27fe0 0x55fad1b3d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f252a8f8640 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b31aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 -- 172.21.15.96:0/1227831664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 msgr2=0x55fad1b31aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b31aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 -- 172.21.15.96:0/1227831664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b27fe0 msgr2=0x55fad1b3d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b27fe0 0x55fad1b3d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 -- 172.21.15.96:0/1227831664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad18666b0 con 0x55fad1b289d0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f25298f6640 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55fad1b29440 tx=0x7f252000a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eed6c222ef8b3ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f250ffff640 1 -- 172.21.15.96:0/1227831664 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520017040 con 0x55fad1b289d0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f250ffff640 1 -- 172.21.15.96:0/1227831664 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2520026070 con 0x55fad1b289d0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.782+0000 7f250ffff640 1 -- 172.21.15.96:0/1227831664 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252000d560 con 0x55fad1b289d0 2024-03-18T15:39:57.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- 172.21.15.96:0/1227831664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 msgr2=0x55fad1b393a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55fad1b29440 tx=0x7f252000a7f0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- 172.21.15.96:0/1227831664 shutdown_connections 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1b31aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b289d0 0x55fad1b393a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 --2- 172.21.15.96:0/1227831664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b27fe0 0x55fad1b3d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- 172.21.15.96:0/1227831664 >> 172.21.15.96:0/1227831664 conn(0x55fad1b1f140 msgr2=0x55fad1b1a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- 172.21.15.96:0/1227831664 shutdown_connections 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- 172.21.15.96:0/1227831664 wait complete. 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 -- start start 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.783+0000 7f252f029740 1 Processor -- start 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1ba2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b3fcc0 con 0x55fad1b27fe0 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b3f5f0 con 0x55fad1b289d0 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fad1b40390 con 0x55fad1b2db40 2024-03-18T15:39:57.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 10 client.0 _ll_register_callbacks cb 0x7f25140035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54016/0 (socket says 172.21.15.96:54016) 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 -- 172.21.15.96:0/1072688001 learned_addr learned my addr 172.21.15.96:0/1072688001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f25298f6640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a8f8640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1ba2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 -- 172.21.15.96:0/1072688001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 msgr2=0x55fad1ba2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1ba2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 -- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 msgr2=0x55fad1b345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b345a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 -- 172.21.15.96:0/1072688001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad1b34bb0 con 0x55fad1b27fe0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252a0f7640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2514005c20 tx=0x7f25140119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b71fdd7c3db536e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f250dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514036040 con 0x55fad1b27fe0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f2514031610 con 0x55fad1b27fe0 2024-03-18T15:39:57.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2520026070 con 0x55fad1b27fe0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.784+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2514009910 con 0x55fad1b27fe0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 10 client.15324 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fad1ba3da0 con 0x55fad1b27fe0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2520026070 con 0x55fad1b27fe0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f251401e420 con 0x55fad1b27fe0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 10 client.15324 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 20 client.15324 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1ba89f0 0x55fad1ba7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 4 client.15324 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "status"}) -- 0x55fad1a8f510 con 0x55fad1ba89f0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f25298f6640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1ba89f0 0x55fad1ba7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f250dffb640 10 client.15324 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:57.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.785+0000 7f25298f6640 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1ba89f0 0x55fad1ba7700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a23d0b4e2291b6 server_cookie=7b50c9037c38ac3f in_seq=0 out_seq=0 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f250dffb640 1 -- 172.21.15.96:0/1072688001 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fad1a8f510 con 0x55fad1ba89f0 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f250dffb640 10 client.15324 handle_command_reply: tid=0 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 client.15324 shutdown 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 msgr2=0x55fad1b34060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2514005c20 tx=0x7f25140119e0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 shutdown_connections 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fad1b2db40 0x55fad1ba2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55fad1ba89f0 0x55fad1ba7700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fad1b289d0 0x55fad1b345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f250dffb640 0 client.15324 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 --2- 172.21.15.96:0/1072688001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad1b27fe0 0x55fad1b34060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 >> 172.21.15.96:0/1072688001 conn(0x55fad1b1f140 msgr2=0x55fad1b3d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 shutdown_connections 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 1 -- 172.21.15.96:0/1072688001 wait complete. 2024-03-18T15:39:57.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252f029740 20 client.15324 trim_cache size 0 max 16384 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24485, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 110.63188964699999, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:57.794 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 119.58168528500001 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 msgr2=0x7f25100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 0x7f25100a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f25140095f0 tx=0x7f2514005f90 comp rx=0 tx=0).stop 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 msgr2=0x7f252419f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f251c037040 tx=0x7f251c032f40 comp rx=0 tx=0).stop 2024-03-18T15:39:57.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 shutdown_connections 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f25100a2800 0x7f25100a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25241019a0 0x7f252419f060 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2524100fe0 0x7f2524197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 --2- 172.21.15.96:0/1679637477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25240ffe90 0x7f2524197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 >> 172.21.15.96:0/1679637477 conn(0x7f25240fb3e0 msgr2=0x7f2524104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 shutdown_connections 2024-03-18T15:39:57.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.786+0000 7f252c35b640 1 -- 172.21.15.96:0/1679637477 wait complete. 2024-03-18T15:39:57.797 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:57 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1580841926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:57.797 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:57 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1525283960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:57.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- 172.21.15.96:0/519430967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601018e0 msgr2=0x7feb60101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 --2- 172.21.15.96:0/519430967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601018e0 0x7feb60101d60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb50009900 tx=0x7feb5002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- 172.21.15.96:0/519430967 shutdown_connections 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 --2- 172.21.15.96:0/519430967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601022a0 0x7feb60108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 --2- 172.21.15.96:0/519430967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601018e0 0x7feb60101d60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 --2- 172.21.15.96:0/519430967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb60100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- 172.21.15.96:0/519430967 >> 172.21.15.96:0/519430967 conn(0x7feb600fbc80 msgr2=0x7feb600fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- 172.21.15.96:0/519430967 shutdown_connections 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- 172.21.15.96:0/519430967 wait complete. 2024-03-18T15:39:57.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 Processor -- start 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.899+0000 7feb64eff640 1 -- start start 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb6006a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601018e0 0x7feb6019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010c370 con 0x7feb601022a0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010c1f0 con 0x7feb601018e0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb6010c4f0 con 0x7feb601006f0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54032/0 (socket says 172.21.15.96:54032) 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 -- 172.21.15.96:0/4039104063 learned_addr learned my addr 172.21.15.96:0/4039104063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5e59c640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb6006a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601018e0 0x7feb6019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 -- 172.21.15.96:0/4039104063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 msgr2=0x7feb6006a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb6006a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 -- 172.21.15.96:0/4039104063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601018e0 msgr2=0x7feb6019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601018e0 0x7feb6019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 -- 172.21.15.96:0/4039104063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6019f740 con 0x7feb601022a0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5e59c640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb6006a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb5ed9d640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb5800eb10 tx=0x7feb5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb4f7fe640 1 -- 172.21.15.96:0/4039104063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58004b20 con 0x7feb601022a0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb4f7fe640 1 -- 172.21.15.96:0/4039104063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb58004cc0 con 0x7feb601022a0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6019fa30 con 0x7feb601022a0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb4f7fe640 1 -- 172.21.15.96:0/4039104063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb58010760 con 0x7feb601022a0 2024-03-18T15:39:57.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.900+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6019ff00 con 0x7feb601022a0 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 Processor -- start 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb4f7fe640 1 -- 172.21.15.96:0/4039104063 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7feb58005020 con 0x7feb601022a0 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 -- start start 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 0x5653d0c8aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c92bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653d0beec80 con 0x5653d0c8b5c0 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d0b97aa0 con 0x5653d0c90730 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb67bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653d0bead20 con 0x5653d0c8abd0 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 0x5653d0c8aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54036/0 (socket says 172.21.15.96:54036) 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 -- 172.21.15.96:0/3029304603 learned_addr learned my addr 172.21.15.96:0/3029304603 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c92bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb4f7fe640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 0x7feb480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5ed9d640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 0x7feb480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 -- 172.21.15.96:0/3029304603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 msgr2=0x5653d0c8aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 0x5653d0c8aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 -- 172.21.15.96:0/3029304603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 msgr2=0x5653d0c92bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c92bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5e59c640 1 -- 172.21.15.96:0/3029304603 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d0bb9ff0 con 0x5653d0c8b5c0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.903+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 0x5653d0c8aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb4f7fe640 1 -- 172.21.15.96:0/4039104063 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feb580cefe0 con 0x7feb601022a0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb540114f0 tx=0x7feb540119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fca8c5dd682a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb4cff9640 1 -- 172.21.15.96:0/3029304603 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54011250 con 0x5653d0c8b5c0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb5ed9d640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 0x7feb480a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb5809b900 tx=0x7feb58066730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb4cff9640 1 -- 172.21.15.96:0/3029304603 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb540035f0 con 0x5653d0c8b5c0 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.904+0000 7feb4cff9640 1 -- 172.21.15.96:0/3029304603 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5400f820 con 0x5653d0c8b5c0 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- 172.21.15.96:0/3029304603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 msgr2=0x5653d0c900c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb540114f0 tx=0x7feb540119b0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- 172.21.15.96:0/3029304603 shutdown_connections 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c92bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8b5c0 0x5653d0c900c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3029304603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8abd0 0x5653d0c8aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- 172.21.15.96:0/3029304603 >> 172.21.15.96:0/3029304603 conn(0x5653d0c86140 msgr2=0x5653d0c86540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- 172.21.15.96:0/3029304603 shutdown_connections 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- 172.21.15.96:0/3029304603 wait complete. 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- start start 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 Processor -- start 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8abd0 0x5653d0c94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653d0c936e0 con 0x5653d0c90730 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d0baaef0 con 0x5653d0c8abd0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653d0c93860 con 0x5653d0c8b5c0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb67bc9740 10 client.0 _ll_register_callbacks cb 0x5653d0ba5dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54052/0 (socket says 172.21.15.96:54052) 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3241515657 learned_addr learned my addr 172.21.15.96:0/3241515657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8abd0 0x5653d0c94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.905+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 msgr2=0x5653d0c952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3241515657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8abd0 msgr2=0x5653d0c94d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8abd0 0x5653d0c94d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3241515657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d0bb7a30 con 0x5653d0c90730 2024-03-18T15:39:57.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb50036040 tx=0x7feb50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48082994ffa4948e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50003c30 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feb50003dd0 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb540035f0 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feb50034440 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 10 client.15336 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653d09032d0 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb540035f0 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7feb500414c0 con 0x5653d0c90730 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 10 client.15336 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 20 client.15336 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.906+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0ca1870 0x5653d0ca2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.907+0000 7feb67bc9740 4 client.15336 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:39:57.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.907+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d0bfc1c0 con 0x5653d0ca1870 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.907+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0ca1870 0x5653d0ca2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.907+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0ca1870 0x5653d0ca2ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424b41f72c83c0ff server_cookie=c2aa0085436b8d9c in_seq=0 out_seq=0 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.907+0000 7feb45ffb640 10 client.15336 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb45ffb640 1 -- 172.21.15.96:0/3241515657 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5653d0bfc1c0 con 0x5653d0ca1870 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb45ffb640 10 client.15336 handle_command_reply: tid=0 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 client.15336 shutdown 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 msgr2=0x5653d0c9c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb50036040 tx=0x7feb50009260 comp rx=0 tx=0).stop 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 shutdown_connections 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0ca1870 0x5653d0ca2ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c90730 0x5653d0c9c300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3241515657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8abd0 0x5653d0c94d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb45ffb640 0 client.15336 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:57.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 >> 172.21.15.96:0/3241515657 conn(0x5653d0c86140 msgr2=0x5653d0c85380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 shutdown_connections 2024-03-18T15:39:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 1 -- 172.21.15.96:0/3241515657 wait complete. 2024-03-18T15:39:57.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.908+0000 7feb67bc9740 20 client.15336 trim_cache size 0 max 16384 2024-03-18T15:39:57.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:57 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1580841926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:57.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:57 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1525283960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:57.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 Processor -- start 2024-03-18T15:39:57.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 -- start start 2024-03-18T15:39:57.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8b5c0 0x5653d0c9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c90730 0x5653d0c94ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca1020 con 0x5653d0c8abd0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca16f0 con 0x5653d0c8b5c0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb67bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca1dc0 con 0x5653d0c90730 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54066/0 (socket says 172.21.15.96:54066) 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3172077619 learned_addr learned my addr 172.21.15.96:0/3172077619 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5e59c640 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8b5c0 0x5653d0c9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c90730 0x5653d0c94ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3172077619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c90730 msgr2=0x5653d0c94ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c90730 0x5653d0c94ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3172077619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8b5c0 msgr2=0x5653d0c9c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8b5c0 0x5653d0c9c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 -- 172.21.15.96:0/3172077619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d09cd6b0 con 0x5653d0c8abd0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb500418e0 tx=0x7feb5002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871ab88d3a2bd5e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb47fff640 1 -- 172.21.15.96:0/3172077619 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb500043e0 con 0x5653d0c8abd0 2024-03-18T15:39:57.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb47fff640 1 -- 172.21.15.96:0/3172077619 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb500057b0 con 0x5653d0c8abd0 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.926+0000 7feb47fff640 1 -- 172.21.15.96:0/3172077619 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50034840 con 0x5653d0c8abd0 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- 172.21.15.96:0/3172077619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 msgr2=0x5653d0c94990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb500418e0 tx=0x7feb5002bc30 comp rx=0 tx=0).stop 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- 172.21.15.96:0/3172077619 shutdown_connections 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c90730 0x5653d0c94ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c8b5c0 0x5653d0c9c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- 172.21.15.96:0/3172077619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c94990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- 172.21.15.96:0/3172077619 >> 172.21.15.96:0/3172077619 conn(0x5653d0c86140 msgr2=0x5653d0ca0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- 172.21.15.96:0/3172077619 shutdown_connections 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- 172.21.15.96:0/3172077619 wait complete. 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- start start 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 Processor -- start 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c97a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca2b60 con 0x5653d0c8abd0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca2490 con 0x5653d0c90730 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5653d0ca3230 con 0x5653d0c8b5c0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb67bc9740 10 client.0 _ll_register_callbacks cb 0x7feb50005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb5dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54068/0 (socket says 172.21.15.96:54068) 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.927+0000 7feb5dd9b640 1 -- 172.21.15.96:0/2018834139 learned_addr learned my addr 172.21.15.96:0/2018834139 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5e59c640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c97a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 -- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 msgr2=0x5653d0c97a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c97a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 -- 172.21.15.96:0/2018834139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 msgr2=0x5653d0c9aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c9aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 -- 172.21.15.96:0/2018834139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d0c9aff0 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb5dd9b640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7feb50041a00 tx=0x7feb50005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd03526a7625b511 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5005f020 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7feb500345e0 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653d0bf6510 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feb5006a050 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 10 client.15342 fetch_fsmap learned FSMap version 24 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5653d0c9c260 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653d0bf6510 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7feb500598b0 con 0x5653d0c8abd0 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 10 client.15342 fetch_fsmap finished waiting for FSMap version 24 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 10 client.15342 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 20 client.15342 populate_metadata read hostname 'smithi096' 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0d0b890 0x5653d0d0a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 4 client.15342 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:39:57.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.928+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "status"}) -- 0x5653d0b74a00 con 0x5653d0d0b890 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb5e59c640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0d0b890 0x5653d0d0a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb45ffb640 10 client.15342 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb5e59c640 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0d0b890 0x5653d0d0a5a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42bb66302f7764ec server_cookie=f03403468cb9ce67 in_seq=0 out_seq=0 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb45ffb640 1 -- 172.21.15.96:0/2018834139 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5653d0b74a00 con 0x5653d0d0b890 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb45ffb640 10 client.15342 handle_command_reply: tid=0 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 client.15342 shutdown 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 msgr2=0x5653d0c974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7feb50041a00 tx=0x7feb50005d50 comp rx=0 tx=0).stop 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 shutdown_connections 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d0c90730 0x5653d0c9aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5653d0c8b5c0 0x5653d0c97a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5653d0d0b890 0x5653d0d0a5a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 --2- 172.21.15.96:0/2018834139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653d0c8abd0 0x5653d0c974f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb45ffb640 0 client.15342 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 >> 172.21.15.96:0/2018834139 conn(0x5653d0c86140 msgr2=0x5653d0c94990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 shutdown_connections 2024-03-18T15:39:57.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 1 -- 172.21.15.96:0/2018834139 wait complete. 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.929+0000 7feb67bc9740 20 client.15342 trim_cache size 0 max 16384 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "419c928c-e53c-11ee-95c9-87774f69a715", 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24493, 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 113.116109103, 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 24, 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 88, 2024-03-18T15:39:57.938 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 86, 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 128.49703994999999 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 msgr2=0x7feb480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 0x7feb480a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7feb5809b900 tx=0x7feb58066730 comp rx=0 tx=0).stop 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 msgr2=0x7feb6019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb5800eb10 tx=0x7feb5800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 shutdown_connections 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb480a2800 0x7feb480a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb601022a0 0x7feb6019ee80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feb601018e0 0x7feb6019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 --2- 172.21.15.96:0/4039104063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb601006f0 0x7feb6006a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 >> 172.21.15.96:0/4039104063 conn(0x7feb600fbc80 msgr2=0x7feb60104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 shutdown_connections 2024-03-18T15:39:57.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:39:57.930+0000 7feb64eff640 1 -- 172.21.15.96:0/4039104063 wait complete. 2024-03-18T15:39:57.941 INFO:teuthology.run_tasks:Running task kclient... 2024-03-18T15:39:57.965 INFO:tasks.kclient:Mounting kernel clients... 2024-03-18T15:39:57.966 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-18T15:39:57.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:39:57.966 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-18T15:39:57.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:39:57.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-03-18T15:39:58.050 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-03-18T15:39:58.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:39:58.052 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-03-18T15:39:58.052 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T15:39:58.052 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T15:39:58.052 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-18T15:39:58.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:57 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1580841926' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:39:58.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:57 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1525283960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:39:58.333 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:39:58.723 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:39:58 smithi096 ceph-mon[31930]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:39:58 smithi193 ceph-mon[36702]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:39:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:39:58 smithi137 ceph-mon[35868]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 -- 172.21.15.96:0/1101433410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 msgr2=0x7f00e4106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 --2- 172.21.15.96:0/1101433410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e4106f10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f00cc0099c0 tx=0x7f00cc009cd0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 -- 172.21.15.96:0/1101433410 shutdown_connections 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 --2- 172.21.15.96:0/1101433410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e41075f0 0x7f00e410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 --2- 172.21.15.96:0/1101433410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e4102510 0x7f00e41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 --2- 172.21.15.96:0/1101433410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e4106f10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 -- 172.21.15.96:0/1101433410 >> 172.21.15.96:0/1101433410 conn(0x7f00e40fdd80 msgr2=0x7f00e41001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 -- 172.21.15.96:0/1101433410 shutdown_connections 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.176+0000 7f00eadd2700 1 -- 172.21.15.96:0/1101433410 wait complete. 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 Processor -- start 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 -- start start 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e4102510 0x7f00e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e41075f0 0x7f00e410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41125b0 con 0x7f00e4102ec0 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4112430 con 0x7f00e4102510 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00eadd2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4112730 con 0x7f00e41075f0 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54090/0 (socket says 172.21.15.96:54090) 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 -- 172.21.15.96:0/3949171398 learned_addr learned my addr 172.21.15.96:0/3949171398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e8b6e700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e4102510 0x7f00e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e936f700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e41075f0 0x7f00e410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 -- 172.21.15.96:0/3949171398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e41075f0 msgr2=0x7f00e410e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e41075f0 0x7f00e410e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 -- 172.21.15.96:0/3949171398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e4102510 msgr2=0x7f00e419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e4102510 0x7f00e419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.177+0000 7f00e3fff700 1 -- 172.21.15.96:0/3949171398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e410eca0 con 0x7f00e4102ec0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00e3fff700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c010 tx=0x7f00cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc027070 con 0x7f00e4102ec0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00cc003730 con 0x7f00e4102ec0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e410ef30 con 0x7f00e4102ec0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc022c70 con 0x7f00e4102ec0 2024-03-18T15:40:00.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.178+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e419b2c0 con 0x7f00e4102ec0 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.179+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e4068f90 con 0x7f00e4102ec0 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.180+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f00cc02a070 con 0x7f00e4102ec0 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.180+0000 7f00e1ffb700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 0x7f00d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.181+0000 7f00e8b6e700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 0x7f00d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.181+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8248+0+0 (secure 0 0 0) 0x7f00cc0cfe20 con 0x7f00e4102ec0 2024-03-18T15:40:00.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.181+0000 7f00e8b6e700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 0x7f00d00a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009550 tx=0x7f00dc0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:00.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.184+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f00cc031070 con 0x7f00e4102ec0 2024-03-18T15:40:00.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.425+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00e419b520 con 0x7f00e4102ec0 2024-03-18T15:40:00.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.426+0000 7f00e1ffb700 1 -- 172.21.15.96:0/3949171398 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f00cc013070 con 0x7f00e4102ec0 2024-03-18T15:40:00.426 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 msgr2=0x7f00d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 0x7f00d00a4d50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009550 tx=0x7f00dc0091b0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 msgr2=0x7f00e419fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00c010 tx=0x7f00cc0047b0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 shutdown_connections 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f00d00a2890 0x7f00d00a4d50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00e41075f0 0x7f00e410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00e4102510 0x7f00e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 --2- 172.21.15.96:0/3949171398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00e4102ec0 0x7f00e419fca0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 >> 172.21.15.96:0/3949171398 conn(0x7f00e40fdd80 msgr2=0x7f00e40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 shutdown_connections 2024-03-18T15:40:00.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:00.428+0000 7f00eadd2700 1 -- 172.21.15.96:0/3949171398 wait complete. 2024-03-18T15:40:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:00 smithi096 ceph-mon[31930]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:40:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:00 smithi096 ceph-mon[31930]: overall HEALTH_OK 2024-03-18T15:40:00.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:00 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3949171398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T15:40:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:00 smithi193 ceph-mon[36702]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:40:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:00 smithi193 ceph-mon[36702]: overall HEALTH_OK 2024-03-18T15:40:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:00 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3949171398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T15:40:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:00 smithi137 ceph-mon[35868]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-18T15:40:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:00 smithi137 ceph-mon[35868]: overall HEALTH_OK 2024-03-18T15:40:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:00 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3949171398' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-18T15:40:01.203 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T15:40:01.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T15:40:01.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-03-18T15:40:01.203 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T15:40:01.204 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:01.204 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T15:40:01.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T15:40:01.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T15:40:01.204 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:40:01.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:01.204 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-03-18T15:40:01.221 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T15:40:01.221 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T15:40:01.221 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-03-18T15:40:01.222 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-18T15:40:01.223 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42129sec preferred_lft 42129sec 2024-03-18T15:40:01.223 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-03-18T15:40:01.223 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:40:01.223 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T15:40:01.223 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:40:01.223 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-18T15:40:01.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-03-18T15:40:01.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-03-18T15:40:01.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-03-18T15:40:01.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T15:40:01.224 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-18T15:40:01.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:01.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:01.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:01.433 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T15:40:01.461 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-03-18T15:40:01.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:01.462 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-03-18T15:40:01.531 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-03-18T15:40:01.531 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T15:40:01.532 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-18T15:40:01.532 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-18T15:40:01.532 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-18T15:40:01.532 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-18T15:40:01.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:01.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:01 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-18T15:40:01.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:01.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:01.851 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-18T15:40:01.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:01.867 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-03-18T15:40:01.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:01 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-18T15:40:01.922 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:40:01.922 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-18T15:40:01.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:40:01.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T15:40:01.922 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-18T15:40:02.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:02.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.044 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T15:40:02.044 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:40:02.044 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-18T15:40:02.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T15:40:02.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T15:40:02.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T15:40:02.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T15:40:02.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T15:40:02.045 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-18T15:40:02.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:02.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:01 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-18T15:40:02.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.187 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:40:02.188 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-03-18T15:40:02.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-03-18T15:40:02.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T15:40:02.188 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-03-18T15:40:02.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:02.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:40:02.269 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:02.283 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:40:02.283 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:40:02.283 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:02.338 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-18T15:40:02.338 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-18T15:40:02.338 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/a620fed7-ceed-43fc-8a14-043aeb386ec1 2024-03-18T15:40:02.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:02.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/a620fed7-ceed-43fc-8a14-043aeb386ec1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-18T15:40:02.422 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-18T15:40:02.422 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-18T15:40:02.422 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:invalid new device string format 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.96:6789,172.21.15.137:6789,172.21.15.193:6789" 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.96:6789,172.21.15.137:6789,172.21.15.193:6789:/volumes/qa/sv_1/a620fed7-ceed-43fc-8a14-043aeb386ec1 2024-03-18T15:40:02.535 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=419c928c-e53c-11ee-95c9-87774f69a715" will pass to kernel 2024-03-18T15:40:02.536 INFO:teuthology.orchestra.run.smithi096.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-18T15:40:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: You just mounted a file system that supports labels which does not 2024-03-18T15:40:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-18T15:40:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: applications will generate AVC messages and not be allowed access to 2024-03-18T15:40:02.536 INFO:teuthology.orchestra.run.smithi096.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-18T15:40:02.538 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-18T15:40:02.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:02.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:02.612 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T15:40:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:02 smithi096 ceph-mon[31930]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:40:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:02 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-18T15:40:02.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:02 smithi096 ceph-mon[31930]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T15:40:02.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T15:40:02.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.709 INFO:teuthology.orchestra.run.smithi096.stdout:client25142 2024-03-18T15:40:02.709 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:40:02.709 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:40:02.766 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T15:40:02.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T15:40:02.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.862 INFO:teuthology.orchestra.run.smithi096.stdout:client25142 2024-03-18T15:40:02.863 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-18T15:40:02.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T15:40:02.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:02 smithi193 ceph-mon[36702]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:40:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:02 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-18T15:40:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:02 smithi193 ceph-mon[36702]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T15:40:02.918 INFO:teuthology.orchestra.run.smithi096.stdout:client25142 2024-03-18T15:40:02.919 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/419c928c-e53c-11ee-95c9-87774f69a715.client25142/status') 2024-03-18T15:40:02.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:40:02.979 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.25142 (3)172.21.15.96:0/2861883833 2024-03-18T15:40:02.979 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-03-18T15:40:02.979 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-03-18T15:40:02.979 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-03-18T15:40:02.979 INFO:teuthology.orchestra.run.smithi096.stderr:71 bytes copied, 2.0239e-05 s, 3.5 MB/s 2024-03-18T15:40:02.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:40:02.987 INFO:teuthology.run_tasks:Running task exec... 2024-03-18T15:40:02.996 INFO:teuthology.task.exec:Executing custom commands... 2024-03-18T15:40:02.997 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:40:02.997 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-18T15:40:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 -- 172.21.15.96:0/541188427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f601022c0 msgr2=0x7f8f60108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 --2- 172.21.15.96:0/541188427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f601022c0 0x7f8f60108b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009ca0 tx=0x7f8f5802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 -- 172.21.15.96:0/541188427 shutdown_connections 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 --2- 172.21.15.96:0/541188427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f601022c0 0x7f8f60108b50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 --2- 172.21.15.96:0/541188427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f60101900 0x7f8f60101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 --2- 172.21.15.96:0/541188427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f60100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 -- 172.21.15.96:0/541188427 >> 172.21.15.96:0/541188427 conn(0x7f8f600fbca0 msgr2=0x7f8f600fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 -- 172.21.15.96:0/541188427 shutdown_connections 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.098+0000 7f8f67061640 1 -- 172.21.15.96:0/541188427 wait complete. 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 Processor -- start 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 -- start start 2024-03-18T15:40:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f601979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f601022c0 0x7f8f6019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010c390 con 0x7f8f60101900 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010c210 con 0x7f8f601022c0 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f67061640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6010c510 con 0x7f8f60100710 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f64dfd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f601979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54110/0 (socket says 172.21.15.96:54110) 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 -- 172.21.15.96:0/669559207 learned_addr learned my addr 172.21.15.96:0/669559207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f655fe640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f601022c0 0x7f8f6019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 -- 172.21.15.96:0/669559207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 msgr2=0x7f8f601979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f601979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 -- 172.21.15.96:0/669559207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f601022c0 msgr2=0x7f8f6019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f601022c0 0x7f8f6019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f57fff640 1 -- 172.21.15.96:0/669559207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6019f600 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.099+0000 7f8f64dfd640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f601979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f57fff640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9f0 tx=0x7f8f5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000cd80 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f50010910 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6019f8f0 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50018990 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.100+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6019fe40 con 0x7f8f60101900 2024-03-18T15:40:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.101+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f601a0070 con 0x7f8f60101900 2024-03-18T15:40:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.102+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f8f50010ac0 con 0x7f8f60101900 2024-03-18T15:40:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.102+0000 7f8f55ffb640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 0x7f8f480a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.103+0000 7f8f64dfd640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 0x7f8f480a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.103+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8f500cede0 con 0x7f8f60101900 2024-03-18T15:40:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.103+0000 7f8f64dfd640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 0x7f8f480a4d60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8f60101760 tx=0x7f8f5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:02 smithi137 ceph-mon[35868]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-18T15:40:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:02 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-18T15:40:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:02 smithi137 ceph-mon[35868]: osdmap e89: 12 total, 12 up, 12 in 2024-03-18T15:40:03.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.107+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f8f5009b800 con 0x7f8f60101900 2024-03-18T15:40:03.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.323+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f8f601a0460 con 0x7f8f60101900 2024-03-18T15:40:03.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.635+0000 7f8f55ffb640 1 -- 172.21.15.96:0/669559207 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f8f5009f8a0 con 0x7f8f60101900 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 msgr2=0x7f8f480a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 0x7f8f480a4d60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8f60101760 tx=0x7f8f5c005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 msgr2=0x7f8f60197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9f0 tx=0x7f8f5000eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 shutdown_connections 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8f480a28a0 0x7f8f480a4d60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f601022c0 0x7f8f6019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f60101900 0x7f8f60197f20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 --2- 172.21.15.96:0/669559207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f60100710 0x7f8f601979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 >> 172.21.15.96:0/669559207 conn(0x7f8f600fbca0 msgr2=0x7f8f600fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:03.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 shutdown_connections 2024-03-18T15:40:03.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.638+0000 7f8f67061640 1 -- 172.21.15.96:0/669559207 wait complete. 2024-03-18T15:40:03.650 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-18T15:40:03.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- 172.21.15.96:0/3165230392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1941033e0 msgr2=0x7ff194109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 --2- 172.21.15.96:0/3165230392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1941033e0 0x7ff194109c70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff184009900 tx=0x7ff18402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- 172.21.15.96:0/3165230392 shutdown_connections 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 --2- 172.21.15.96:0/3165230392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1941033e0 0x7ff194109c70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 --2- 172.21.15.96:0/3165230392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff194102a20 0x7ff194102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 --2- 172.21.15.96:0/3165230392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194101830 0x7ff194101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- 172.21.15.96:0/3165230392 >> 172.21.15.96:0/3165230392 conn(0x7ff1940fcda0 msgr2=0x7ff1940ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- 172.21.15.96:0/3165230392 shutdown_connections 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- 172.21.15.96:0/3165230392 wait complete. 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 Processor -- start 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.754+0000 7ff19bb77640 1 -- start start 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102a20 0x7ff194197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1941033e0 0x7ff19419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410d4b0 con 0x7ff194101830 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410d330 con 0x7ff194102a20 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410d630 con 0x7ff1941033e0 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40126/0 (socket says 172.21.15.96:40126) 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 -- 172.21.15.96:0/1843481217 learned_addr learned my addr 172.21.15.96:0/1843481217 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199112640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102a20 0x7ff194197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19a114640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1941033e0 0x7ff19419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 -- 172.21.15.96:0/1843481217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1941033e0 msgr2=0x7ff19419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1941033e0 0x7ff19419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 -- 172.21.15.96:0/1843481217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102a20 msgr2=0x7ff194197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102a20 0x7ff194197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 -- 172.21.15.96:0/1843481217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19419f650 con 0x7ff194101830 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff199913640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00ebc0 tx=0x7ff18c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003b70 con 0x7ff194101830 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19419f940 con 0x7ff194101830 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18c003d10 con 0x7ff194101830 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c019980 con 0x7ff194101830 2024-03-18T15:40:03.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.755+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19419fe90 con 0x7ff194101830 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.756+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff194108a80 con 0x7ff194101830 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.757+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7ff18c019b20 con 0x7ff194101830 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.758+0000 7ff192ffd640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 0x7ff17c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.758+0000 7ff199112640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 0x7ff17c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.758+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff18c0cf270 con 0x7ff194101830 2024-03-18T15:40:03.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.758+0000 7ff199112640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 0x7ff17c0a4d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff194198f40 tx=0x7ff188005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:03.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.762+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff18c09c2e0 con 0x7ff194101830 2024-03-18T15:40:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:03 smithi193 ceph-mon[36702]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T15:40:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:03 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T15:40:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:03 smithi193 ceph-mon[36702]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-18T15:40:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-mon[31930]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T15:40:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T15:40:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-mon[31930]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-18T15:40:03.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.966+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ff194061cc0 con 0x7ff17c0a28a0 2024-03-18T15:40:03.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.984+0000 7ff192ffd640 1 -- 172.21.15.96:0/1843481217 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ff194061cc0 con 0x7ff17c0a28a0 2024-03-18T15:40:03.985 INFO:teuthology.orchestra.run.smithi096.stdout:{} 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 msgr2=0x7ff17c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 0x7ff17c0a4d60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff194198f40 tx=0x7ff188005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 msgr2=0x7ff194197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00ebc0 tx=0x7ff18c00c6a0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 shutdown_connections 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff17c0a28a0 0x7ff17c0a4d60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1941033e0 0x7ff19419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff194102a20 0x7ff194197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 --2- 172.21.15.96:0/1843481217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff194101830 0x7ff194197a30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 >> 172.21.15.96:0/1843481217 conn(0x7ff1940fcda0 msgr2=0x7ff194105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 shutdown_connections 2024-03-18T15:40:03.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:03.986+0000 7ff19bb77640 1 -- 172.21.15.96:0/1843481217 wait complete. 2024-03-18T15:40:03.997 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-18T15:40:04.008 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-18T15:40:04.040 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:40:04.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:03 smithi137 ceph-mon[35868]: osdmap e90: 12 total, 12 up, 12 in 2024-03-18T15:40:04.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:03 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-18T15:40:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:03 smithi137 ceph-mon[35868]: pgmap v351: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 127 B/s wr, 8 op/s 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- 172.21.15.96:0/3716223034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75581022e0 msgr2=0x7f75581046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 --2- 172.21.15.96:0/3716223034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75581022e0 0x7f75581046d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7548009900 tx=0x7f754802bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- 172.21.15.96:0/3716223034 shutdown_connections 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 --2- 172.21.15.96:0/3716223034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7558107540 0x7f7558109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 --2- 172.21.15.96:0/3716223034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558104c10 0x7f7558107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 --2- 172.21.15.96:0/3716223034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75581022e0 0x7f75581046d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- 172.21.15.96:0/3716223034 >> 172.21.15.96:0/3716223034 conn(0x7f75580fbc70 msgr2=0x7f75580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- 172.21.15.96:0/3716223034 shutdown_connections 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- 172.21.15.96:0/3716223034 wait complete. 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 Processor -- start 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.143+0000 7f755e061640 1 -- start start 2024-03-18T15:40:04.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75581022e0 0x7f7558197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104c10 0x7f7558198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d4a0 con 0x7f7558107540 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d320 con 0x7f7558104c10 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d620 con 0x7f75581022e0 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f75577fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75581022e0 0x7f7558197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40170/0 (socket says 172.21.15.96:40170) 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 -- 172.21.15.96:0/1904624846 learned_addr learned my addr 172.21.15.96:0/1904624846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f754effd640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104c10 0x7f7558198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 -- 172.21.15.96:0/1904624846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75581022e0 msgr2=0x7f7558197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75581022e0 0x7f7558197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 -- 172.21.15.96:0/1904624846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104c10 msgr2=0x7f7558198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104c10 0x7f7558198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 -- 172.21.15.96:0/1904624846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755819f770 con 0x7f7558107540 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f7557fff640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f755000eb10 tx=0x7f755000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550004b20 con 0x7f7558107540 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7550004cc0 con 0x7f7558107540 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755819fa60 con 0x7f7558107540 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550010760 con 0x7f7558107540 2024-03-18T15:40:04.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.144+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755819ffb0 con 0x7f7558107540 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.146+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75580fd760 con 0x7f7558107540 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.147+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f7550005000 con 0x7f7558107540 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.147+0000 7f75557fa640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 0x7f75440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.147+0000 7f75577fe640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 0x7f75440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.148+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f75500cf4e0 con 0x7f7558107540 2024-03-18T15:40:04.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.148+0000 7f75577fe640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 0x7f75440a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f754802c7b0 tx=0x7f7548005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.151+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f755009c550 con 0x7f7558107540 2024-03-18T15:40:04.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.330+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75580623e0 con 0x7f7558107540 2024-03-18T15:40:04.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.331+0000 7f75557fa640 1 -- 172.21.15.96:0/1904624846 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22715 (secure 0 0 0) 0x7f755009be50 con 0x7f7558107540 2024-03-18T15:40:04.332 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:04.332 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":90,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:40:02.631764+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:40:04.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 msgr2=0x7f75440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 0x7f75440a4cc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f754802c7b0 tx=0x7f7548005fd0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 msgr2=0x7f755819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f755000eb10 tx=0x7f755000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 shutdown_connections 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75440a2800 0x7f75440a4cc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7558107540 0x7f755819f100 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7558104c10 0x7f7558198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 --2- 172.21.15.96:0/1904624846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75581022e0 0x7f7558197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.333+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 >> 172.21.15.96:0/1904624846 conn(0x7f75580fbc70 msgr2=0x7f7558102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.334+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 shutdown_connections 2024-03-18T15:40:04.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.334+0000 7f755e061640 1 -- 172.21.15.96:0/1904624846 wait complete. 2024-03-18T15:40:04.344 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:40:03.971+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:40:04.344 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:40:03.971+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:40:04.344 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:40:03.971+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:40:04.344 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:40:03.971+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:40:04.345 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:40:03 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:40:03.971+0000 7fae5c07c700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-18T15:40:04.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:40:04.346 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 -- 172.21.15.96:0/832587302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 msgr2=0x7f0220102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 --2- 172.21.15.96:0/832587302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220102e60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0210009900 tx=0x7f021002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 -- 172.21.15.96:0/832587302 shutdown_connections 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 --2- 172.21.15.96:0/832587302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201033a0 0x7f0220109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 --2- 172.21.15.96:0/832587302 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220102e60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 --2- 172.21.15.96:0/832587302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201017f0 0x7f0220101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 -- 172.21.15.96:0/832587302 >> 172.21.15.96:0/832587302 conn(0x7f02200fcd80 msgr2=0x7f02200ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 -- 172.21.15.96:0/832587302 shutdown_connections 2024-03-18T15:40:04.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.450+0000 7f0224d1b640 1 -- 172.21.15.96:0/832587302 wait complete. 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 Processor -- start 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 -- start start 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201017f0 0x7f0220197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201033a0 0x7f022019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f022010d470 con 0x7f02201029e0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f022010d2f0 con 0x7f02201017f0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f0224d1b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f022010d5f0 con 0x7f02201033a0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40182/0 (socket says 172.21.15.96:40182) 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 -- 172.21.15.96:0/2157638944 learned_addr learned my addr 172.21.15.96:0/2157638944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021ed9d640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201033a0 0x7f022019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021e59c640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201017f0 0x7f0220197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 -- 172.21.15.96:0/2157638944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201033a0 msgr2=0x7f022019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201033a0 0x7f022019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 -- 172.21.15.96:0/2157638944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201017f0 msgr2=0x7f0220197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201017f0 0x7f0220197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 -- 172.21.15.96:0/2157638944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022019f730 con 0x7f02201029e0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f021dd9b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0210009230 tx=0x7f021002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.451+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210041070 con 0x7f02201029e0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.452+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022019f9c0 con 0x7f02201029e0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.452+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0210004450 con 0x7f02201029e0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.452+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210003e20 con 0x7f02201029e0 2024-03-18T15:40:04.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.452+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022019ff00 con 0x7f02201029e0 2024-03-18T15:40:04.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.453+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0220108be0 con 0x7f02201029e0 2024-03-18T15:40:04.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.454+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 137290+0+0 (secure 0 0 0) 0x7f0210044070 con 0x7f02201029e0 2024-03-18T15:40:04.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.454+0000 7f020f7fe640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 0x7f02080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.455+0000 7f021e59c640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 0x7f02080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.455+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f02100f0620 con 0x7f02201029e0 2024-03-18T15:40:04.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.455+0000 7f021e59c640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 0x7f02080a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0220102840 tx=0x7f0214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.459+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f02100bd040 con 0x7f02201029e0 2024-03-18T15:40:04.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.634+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f02200623e0 con 0x7f02201029e0 2024-03-18T15:40:04.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.634+0000 7f020f7fe640 1 -- 172.21.15.96:0/2157638944 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f02100c10e0 con 0x7f02201029e0 2024-03-18T15:40:04.635 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 msgr2=0x7f02080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 0x7f02080a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0220102840 tx=0x7f0214005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 msgr2=0x7f0220198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0210009230 tx=0x7f021002c650 comp rx=0 tx=0).stop 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 shutdown_connections 2024-03-18T15:40:04.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f02080a2800 0x7f02080a4cc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02201033a0 0x7f022019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02201029e0 0x7f0220198050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 --2- 172.21.15.96:0/2157638944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02201017f0 0x7f0220197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 >> 172.21.15.96:0/2157638944 conn(0x7f02200fcd80 msgr2=0x7f0220105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 shutdown_connections 2024-03-18T15:40:04.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.636+0000 7f0224d1b640 1 -- 172.21.15.96:0/2157638944 wait complete. 2024-03-18T15:40:04.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:40:04.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- 172.21.15.96:0/321520790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 msgr2=0x7f4d4c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/321520790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c100b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009900 tx=0x7f4d4002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:04.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- 172.21.15.96:0/321520790 shutdown_connections 2024-03-18T15:40:04.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/321520790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c102290 0x7f4d4c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/321520790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c1018d0 0x7f4d4c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/321520790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c100b00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- 172.21.15.96:0/321520790 >> 172.21.15.96:0/321520790 conn(0x7f4d4c0fbc70 msgr2=0x7f4d4c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- 172.21.15.96:0/321520790 shutdown_connections 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- 172.21.15.96:0/321520790 wait complete. 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 Processor -- start 2024-03-18T15:40:04.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 -- start start 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c1018d0 0x7f4d4c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.758+0000 7f4d53d5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 0x7f4d4c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d53d5b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10c330 con 0x7f4d4c1006e0 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d53d5b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10c1b0 con 0x7f4d4c1018d0 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d53d5b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c10c4b0 con 0x7f4d4c102290 2024-03-18T15:40:04.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40196/0 (socket says 172.21.15.96:40196) 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 -- 172.21.15.96:0/3407454585 learned_addr learned my addr 172.21.15.96:0/3407454585 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d512f6640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c1018d0 0x7f4d4c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d522f8640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 0x7f4d4c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 -- 172.21.15.96:0/3407454585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 msgr2=0x7f4d4c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 0x7f4d4c19e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 -- 172.21.15.96:0/3407454585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c1018d0 msgr2=0x7f4d4c196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c1018d0 0x7f4d4c196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 -- 172.21.15.96:0/3407454585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19e670 con 0x7f4d4c1006e0 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d522f8640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 0x7f4d4c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d51af7640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000c00 tx=0x7f4d4002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40041070 con 0x7f4d4c1006e0 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40004450 con 0x7f4d4c1006e0 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40034440 con 0x7f4d4c1006e0 2024-03-18T15:40:04.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19e900 con 0x7f4d4c1006e0 2024-03-18T15:40:04.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.759+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c19ee40 con 0x7f4d4c1006e0 2024-03-18T15:40:04.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.760+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c107b20 con 0x7f4d4c1006e0 2024-03-18T15:40:04.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.761+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f4d40044070 con 0x7f4d4c1006e0 2024-03-18T15:40:04.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.762+0000 7f4d3effd640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 0x7f4d380a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:04.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.762+0000 7f4d512f6640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 0x7f4d380a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:04.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.762+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4d400f0620 con 0x7f4d4c1006e0 2024-03-18T15:40:04.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.762+0000 7f4d512f6640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 0x7f4d380a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c197f60 tx=0x7f4d48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:04.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.766+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f4d400bd6d0 con 0x7f4d4c1006e0 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1904624846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:04 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2157638944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1904624846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:04 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2157638944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:04.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.940+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d4c02d1b0 con 0x7f4d4c1006e0 2024-03-18T15:40:04.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.940+0000 7f4d3effd640 1 -- 172.21.15.96:0/3407454585 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f4d400bcfd0 con 0x7f4d4c1006e0 2024-03-18T15:40:04.942 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:40:04.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 msgr2=0x7f4d380a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 0x7f4d380a4dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c197f60 tx=0x7f4d48005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 msgr2=0x7f4d4c196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000c00 tx=0x7f4d4002c640 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 shutdown_connections 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4d380a2910 0x7f4d380a4dd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d4c102290 0x7f4d4c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d4c1018d0 0x7f4d4c196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 --2- 172.21.15.96:0/3407454585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4c1006e0 0x7f4d4c196a50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 >> 172.21.15.96:0/3407454585 conn(0x7f4d4c0fbc70 msgr2=0x7f4d4c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 shutdown_connections 2024-03-18T15:40:04.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:04.943+0000 7f4d53d5b640 1 -- 172.21.15.96:0/3407454585 wait complete. 2024-03-18T15:40:04.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 -- 172.21.15.96:0/512829424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641033a0 msgr2=0x7f4464109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 --2- 172.21.15.96:0/512829424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641033a0 0x7f4464109c30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f445c009ca0 tx=0x7f445c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 -- 172.21.15.96:0/512829424 shutdown_connections 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 --2- 172.21.15.96:0/512829424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641033a0 0x7f4464109c30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 --2- 172.21.15.96:0/512829424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641029e0 0x7f4464102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 --2- 172.21.15.96:0/512829424 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641017f0 0x7f4464101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 -- 172.21.15.96:0/512829424 >> 172.21.15.96:0/512829424 conn(0x7f44640fcd80 msgr2=0x7f44640ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 -- 172.21.15.96:0/512829424 shutdown_connections 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 -- 172.21.15.96:0/512829424 wait complete. 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.058+0000 7f44690bf640 1 Processor -- start 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 -- start start 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641029e0 0x7f4464073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641033a0 0x7f4464074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f446410d470 con 0x7f44641017f0 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f446410d2f0 con 0x7f44641029e0 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f446410d5f0 con 0x7f44641033a0 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40210/0 (socket says 172.21.15.96:40210) 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 -- 172.21.15.96:0/708120109 learned_addr learned my addr 172.21.15.96:0/708120109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:05.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f445bfff640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641029e0 0x7f4464073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f446359e640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641033a0 0x7f4464074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 -- 172.21.15.96:0/708120109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641033a0 msgr2=0x7f4464074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641033a0 0x7f4464074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 -- 172.21.15.96:0/708120109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641029e0 msgr2=0x7f4464073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641029e0 0x7f4464073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 -- 172.21.15.96:0/708120109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44640749a0 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4462d9d640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f445400e9c0 tx=0x7f445400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445400cd80 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44641a1bd0 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44540035f0 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.059+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454010690 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.060+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a2120 con 0x7f44641017f0 2024-03-18T15:40:05.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.060+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4464108b10 con 0x7f44641017f0 2024-03-18T15:40:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.061+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f44540037a0 con 0x7f44641017f0 2024-03-18T15:40:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.062+0000 7f4460d99640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 0x7f44500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.062+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f44540cf060 con 0x7f44641017f0 2024-03-18T15:40:05.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.062+0000 7f445bfff640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 0x7f44500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.062+0000 7f445bfff640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 0x7f44500a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4464075300 tx=0x7f444c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.067+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f445409bf70 con 0x7f44641017f0 2024-03-18T15:40:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T15:40:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/669559207' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-18T15:40:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:40:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-18T15:40:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1904624846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:04 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2157638944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.243+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f446402d3f0 con 0x7f44641017f0 2024-03-18T15:40:05.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.243+0000 7f4460d99640 1 -- 172.21.15.96:0/708120109 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f445409b870 con 0x7f44641017f0 2024-03-18T15:40:05.244 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:40:05.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 msgr2=0x7f44500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 0x7f44500a4d50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4464075300 tx=0x7f444c005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:05.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 msgr2=0x7f4464075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f445400e9c0 tx=0x7f445400ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 shutdown_connections 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44500a2890 0x7f44500a4d50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44641033a0 0x7f4464074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44641029e0 0x7f4464073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 --2- 172.21.15.96:0/708120109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44641017f0 0x7f4464075590 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 >> 172.21.15.96:0/708120109 conn(0x7f44640fcd80 msgr2=0x7f4464105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 shutdown_connections 2024-03-18T15:40:05.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.245+0000 7f44690bf640 1 -- 172.21.15.96:0/708120109 wait complete. 2024-03-18T15:40:05.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 -- 172.21.15.96:0/1409309956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b54100700 msgr2=0x7f2b54100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1409309956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b54100700 0x7f2b54100b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009900 tx=0x7f2b4402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 -- 172.21.15.96:0/1409309956 shutdown_connections 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1409309956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 0x7f2b54108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1409309956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b541018f0 0x7f2b54101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1409309956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b54100700 0x7f2b54100b20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 -- 172.21.15.96:0/1409309956 >> 172.21.15.96:0/1409309956 conn(0x7f2b540fbc90 msgr2=0x7f2b540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 -- 172.21.15.96:0/1409309956 shutdown_connections 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 -- 172.21.15.96:0/1409309956 wait complete. 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.358+0000 7f2b597bc640 1 Processor -- start 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- start start 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b54100700 0x7f2b5406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 0x7f2b5419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410c380 con 0x7f2b541018f0 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410c200 con 0x7f2b54100700 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5410c500 con 0x7f2b541022b0 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40224/0 (socket says 172.21.15.96:40224) 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 -- 172.21.15.96:0/1712776257 learned_addr learned my addr 172.21.15.96:0/1712776257 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b52ffd640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b54100700 0x7f2b5406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b537fe640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 0x7f2b5419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 -- 172.21.15.96:0/1712776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 msgr2=0x7f2b5419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 0x7f2b5419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 -- 172.21.15.96:0/1712776257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b54100700 msgr2=0x7f2b5406a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b54100700 0x7f2b5406a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 -- 172.21.15.96:0/1712776257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5419f720 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b4bfff640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00e9f0 tx=0x7f2b3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c00cd80 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3c010910 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5419fa10 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c018990 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.359+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5419ffb0 con 0x7f2b541018f0 2024-03-18T15:40:05.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.360+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b54107a20 con 0x7f2b541018f0 2024-03-18T15:40:05.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.362+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f2b3c010ac0 con 0x7f2b541018f0 2024-03-18T15:40:05.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.362+0000 7f2b50ff9640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 0x7f2b400a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.362+0000 7f2b52ffd640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 0x7f2b400a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.362+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2b3c0cf2b0 con 0x7f2b541018f0 2024-03-18T15:40:05.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.363+0000 7f2b52ffd640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 0x7f2b400a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.366+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2b3c09bb70 con 0x7f2b541018f0 2024-03-18T15:40:05.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.542+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b5402d400 con 0x7f2b541018f0 2024-03-18T15:40:05.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.543+0000 7f2b50ff9640 1 -- 172.21.15.96:0/1712776257 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22715 (secure 0 0 0) 0x7f2b3c09fc10 con 0x7f2b541018f0 2024-03-18T15:40:05.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:05.545 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":90,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:40:02.631764+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:40:05.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.545+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 msgr2=0x7f2b400a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.545+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 0x7f2b400a4dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b44005fd0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.545+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 msgr2=0x7f2b5419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.545+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00e9f0 tx=0x7f2b3c00eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 shutdown_connections 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2b400a2910 0x7f2b400a4dd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b541022b0 0x7f2b5419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b541018f0 0x7f2b5419c820 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 --2- 172.21.15.96:0/1712776257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b54100700 0x7f2b5406a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 >> 172.21.15.96:0/1712776257 conn(0x7f2b540fbc90 msgr2=0x7f2b54104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 shutdown_connections 2024-03-18T15:40:05.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.546+0000 7f2b597bc640 1 -- 172.21.15.96:0/1712776257 wait complete. 2024-03-18T15:40:05.558 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:40:05.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 -- 172.21.15.96:0/1253292193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41022a0 msgr2=0x7f65f4108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/1253292193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41022a0 0x7f65f4108b30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ca0 tx=0x7f65ec02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 -- 172.21.15.96:0/1253292193 shutdown_connections 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/1253292193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41022a0 0x7f65f4108b30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/1253292193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41018e0 0x7f65f4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/1253292193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41006f0 0x7f65f4100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 -- 172.21.15.96:0/1253292193 >> 172.21.15.96:0/1253292193 conn(0x7f65f40fbc80 msgr2=0x7f65f40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 -- 172.21.15.96:0/1253292193 shutdown_connections 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.662+0000 7f65fb4a1640 1 -- 172.21.15.96:0/1253292193 wait complete. 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 Processor -- start 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 -- start start 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41006f0 0x7f65f4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41022a0 0x7f65f419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410d450 con 0x7f65f41018e0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410d2d0 con 0x7f65f41006f0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65fb4a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65f410d5d0 con 0x7f65f41022a0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40238/0 (socket says 172.21.15.96:40238) 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 -- 172.21.15.96:0/3688969445 learned_addr learned my addr 172.21.15.96:0/3688969445 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f923d640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41006f0 0x7f65f4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f9a3e640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41022a0 0x7f65f419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 -- 172.21.15.96:0/3688969445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41022a0 msgr2=0x7f65f419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41022a0 0x7f65f419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 -- 172.21.15.96:0/3688969445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41006f0 msgr2=0x7f65f4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41006f0 0x7f65f4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 -- 172.21.15.96:0/3688969445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f419f6e0 con 0x7f65f41018e0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.663+0000 7f65f8a3c640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.664+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400cd80 con 0x7f65f41018e0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.664+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4010910 con 0x7f65f41018e0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.664+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f419f9d0 con 0x7f65f41018e0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.664+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4018990 con 0x7f65f41018e0 2024-03-18T15:40:05.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.664+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f419ff20 con 0x7f65f41018e0 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.665+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f4068a20 con 0x7f65f41018e0 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.666+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f65e4010ac0 con 0x7f65f41018e0 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.666+0000 7f65ea7fc640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 0x7f65e00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.667+0000 7f65f923d640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 0x7f65e00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.667+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f65e40cef70 con 0x7f65f41018e0 2024-03-18T15:40:05.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.667+0000 7f65f923d640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 0x7f65e00a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65f4101740 tx=0x7f65f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.671+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f65e409b830 con 0x7f65f41018e0 2024-03-18T15:40:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.848+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65f402d1e0 con 0x7f65f41018e0 2024-03-18T15:40:05.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.848+0000 7f65ea7fc640 1 -- 172.21.15.96:0/3688969445 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f65e409f8d0 con 0x7f65f41018e0 2024-03-18T15:40:05.849 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 msgr2=0x7f65e00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 0x7f65e00a4d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65f4101740 tx=0x7f65f0005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 msgr2=0x7f65f4198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f65e400e9f0 tx=0x7f65e400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 shutdown_connections 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65e00a2890 0x7f65e00a4d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65f41022a0 0x7f65f419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65f41018e0 0x7f65f4198000 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 --2- 172.21.15.96:0/3688969445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65f41006f0 0x7f65f4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 >> 172.21.15.96:0/3688969445 conn(0x7f65f40fbc80 msgr2=0x7f65f4104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 shutdown_connections 2024-03-18T15:40:05.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.850+0000 7f65fb4a1640 1 -- 172.21.15.96:0/3688969445 wait complete. 2024-03-18T15:40:05.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:05 smithi137 ceph-mon[35868]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T15:40:05.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3407454585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/708120109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:05 smithi137 ceph-mon[35868]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-03-18T15:40:05.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:05 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1712776257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:05.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:05 smithi096 ceph-mon[31930]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T15:40:05.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3407454585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/708120109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:05 smithi096 ceph-mon[31930]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-03-18T15:40:05.861 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:05 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1712776257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:05.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:40:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:05 smithi193 ceph-mon[36702]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-18T15:40:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3407454585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/708120109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:05 smithi193 ceph-mon[36702]: pgmap v352: 97 pgs: 1 peering, 96 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 14 op/s 2024-03-18T15:40:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:05 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1712776257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 -- 172.21.15.96:0/358921255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 msgr2=0x7fb91c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 --2- 172.21.15.96:0/358921255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c101d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009900 tx=0x7fb90c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 -- 172.21.15.96:0/358921255 shutdown_connections 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 --2- 172.21.15.96:0/358921255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c1022b0 0x7fb91c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 --2- 172.21.15.96:0/358921255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c101d70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 --2- 172.21.15.96:0/358921255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c100700 0x7fb91c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 -- 172.21.15.96:0/358921255 >> 172.21.15.96:0/358921255 conn(0x7fb91c0fbc90 msgr2=0x7fb91c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 -- 172.21.15.96:0/358921255 shutdown_connections 2024-03-18T15:40:05.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.965+0000 7fb92351d640 1 -- 172.21.15.96:0/358921255 wait complete. 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 Processor -- start 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 -- start start 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 0x7fb91c104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c1022b0 0x7fb91c1032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c10c380 con 0x7fb91c1018f0 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c10c200 con 0x7fb91c1022b0 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb92351d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c10c500 con 0x7fb91c100700 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb920ab8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb920ab8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40248/0 (socket says 172.21.15.96:40248) 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb920ab8640 1 -- 172.21.15.96:0/4232405653 learned_addr learned my addr 172.21.15.96:0/4232405653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb9212b9640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 0x7fb91c104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.966+0000 7fb921aba640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c1022b0 0x7fb91c1032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 -- 172.21.15.96:0/4232405653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 msgr2=0x7fb91c104760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 0x7fb91c104760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 -- 172.21.15.96:0/4232405653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c1022b0 msgr2=0x7fb91c1032e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c1022b0 0x7fb91c1032e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 -- 172.21.15.96:0/4232405653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c103b70 con 0x7fb91c1018f0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb9212b9640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 0x7fb91c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb920ab8640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009230 tx=0x7fb90c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c041070 con 0x7fb91c1018f0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c1a1bf0 con 0x7fb91c1018f0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb90c004450 con 0x7fb91c1018f0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c003e20 con 0x7fb91c1018f0 2024-03-18T15:40:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.967+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c1a2140 con 0x7fb91c1018f0 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.968+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91c1a2310 con 0x7fb91c1018f0 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.969+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fb90c044070 con 0x7fb91c1018f0 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.969+0000 7fb9127fc640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 0x7fb9080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.970+0000 7fb9212b9640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 0x7fb9080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.970+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb90c0f0690 con 0x7fb91c1018f0 2024-03-18T15:40:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.970+0000 7fb9212b9640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 0x7fb9080a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb91c101750 tx=0x7fb918009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:05.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:05.974+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb90c0bd060 con 0x7fb91c1018f0 2024-03-18T15:40:06.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.146+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb91c1a24e0 con 0x7fb91c1018f0 2024-03-18T15:40:06.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.146+0000 7fb9127fc640 1 -- 172.21.15.96:0/4232405653 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fb90c0c1100 con 0x7fb91c1018f0 2024-03-18T15:40:06.147 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:40:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 msgr2=0x7fb9080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 0x7fb9080a4d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb91c101750 tx=0x7fb918009500 comp rx=0 tx=0).stop 2024-03-18T15:40:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 msgr2=0x7fb91c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009230 tx=0x7fb90c02c650 comp rx=0 tx=0).stop 2024-03-18T15:40:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 shutdown_connections 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9080a2890 0x7fb9080a4d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb91c1022b0 0x7fb91c1032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb91c1018f0 0x7fb91c102da0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 --2- 172.21.15.96:0/4232405653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb91c100700 0x7fb91c104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 >> 172.21.15.96:0/4232405653 conn(0x7fb91c0fbc90 msgr2=0x7fb91c069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 shutdown_connections 2024-03-18T15:40:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.148+0000 7fb92351d640 1 -- 172.21.15.96:0/4232405653 wait complete. 2024-03-18T15:40:06.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:40:06.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 -- 172.21.15.96:0/1866946814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 msgr2=0x7fb098109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 --2- 172.21.15.96:0/1866946814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098109c30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb090009ca0 tx=0x7fb09002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 -- 172.21.15.96:0/1866946814 shutdown_connections 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 --2- 172.21.15.96:0/1866946814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098109c30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 --2- 172.21.15.96:0/1866946814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 0x7fb098102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 --2- 172.21.15.96:0/1866946814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 -- 172.21.15.96:0/1866946814 >> 172.21.15.96:0/1866946814 conn(0x7fb0980fcd80 msgr2=0x7fb0980ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 -- 172.21.15.96:0/1866946814 shutdown_connections 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.264+0000 7fb09fc34640 1 -- 172.21.15.96:0/1866946814 wait complete. 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 Processor -- start 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 -- start start 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098075590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 0x7fb098073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810d470 con 0x7fb0981033a0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810d2f0 con 0x7fb0981017f0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09fc34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb09810d5f0 con 0x7fb0981029e0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09e1d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09e1d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40270/0 (socket says 172.21.15.96:40270) 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09e1d1640 1 -- 172.21.15.96:0/194543946 learned_addr learned my addr 172.21.15.96:0/194543946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d1cf640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 0x7fb098073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 -- 172.21.15.96:0/194543946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 msgr2=0x7fb098073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 0x7fb098073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 -- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 msgr2=0x7fb098074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 -- 172.21.15.96:0/194543946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0980749a0 con 0x7fb0981017f0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09e1d1640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.265+0000 7fb09d9d0640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098075590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb09400e9c0 tx=0x7fb09400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09400ccc0 con 0x7fb0981017f0 2024-03-18T15:40:06.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0981a1bd0 con 0x7fb0981017f0 2024-03-18T15:40:06.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0940035f0 con 0x7fb0981017f0 2024-03-18T15:40:06.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09400ce60 con 0x7fb0981017f0 2024-03-18T15:40:06.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981a2170 con 0x7fb0981017f0 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.266+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb098108b10 con 0x7fb0981017f0 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.268+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fb0940037a0 con 0x7fb0981017f0 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.268+0000 7fb08effd640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 0x7fb0840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.268+0000 7fb09d1cf640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 0x7fb0840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.268+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb0940cf200 con 0x7fb0981017f0 2024-03-18T15:40:06.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.269+0000 7fb09d1cf640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 0x7fb0840a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb098075300 tx=0x7fb088005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.273+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb09409c140 con 0x7fb0981017f0 2024-03-18T15:40:06.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.447+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb09802d420 con 0x7fb0981017f0 2024-03-18T15:40:06.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.448+0000 7fb08effd640 1 -- 172.21.15.96:0/194543946 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb09409ba40 con 0x7fb0981017f0 2024-03-18T15:40:06.449 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 msgr2=0x7fb0840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 0x7fb0840a4d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb098075300 tx=0x7fb088005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 msgr2=0x7fb098075590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098075590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb09400e9c0 tx=0x7fb09400ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 shutdown_connections 2024-03-18T15:40:06.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb0840a2890 0x7fb0840a4d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0981033a0 0x7fb098074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0981029e0 0x7fb098073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 --2- 172.21.15.96:0/194543946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0981017f0 0x7fb098075590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 >> 172.21.15.96:0/194543946 conn(0x7fb0980fcd80 msgr2=0x7fb098105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 shutdown_connections 2024-03-18T15:40:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.450+0000 7fb09fc34640 1 -- 172.21.15.96:0/194543946 wait complete. 2024-03-18T15:40:06.461 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-18T15:40:06.462 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 -- 172.21.15.96:0/532297064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 msgr2=0x7fd75c107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 --2- 172.21.15.96:0/532297064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c107040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd744009900 tx=0x7fd74402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 -- 172.21.15.96:0/532297064 shutdown_connections 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 --2- 172.21.15.96:0/532297064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 0x7fd75c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 --2- 172.21.15.96:0/532297064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c107040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 --2- 172.21.15.96:0/532297064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 -- 172.21.15.96:0/532297064 >> 172.21.15.96:0/532297064 conn(0x7fd75c0fbc90 msgr2=0x7fd75c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 -- 172.21.15.96:0/532297064 shutdown_connections 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.563+0000 7fd761b22640 1 -- 172.21.15.96:0/532297064 wait complete. 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 Processor -- start 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 -- start start 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 0x7fd75c100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10d490 con 0x7fd75c104c50 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10d310 con 0x7fd75c107580 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd761b22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10d610 con 0x7fd75c102320 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40280/0 (socket says 172.21.15.96:40280) 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd752ffd640 1 -- 172.21.15.96:0/602099130 learned_addr learned my addr 172.21.15.96:0/602099130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.564+0000 7fd75bfff640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 0x7fd75c100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 -- 172.21.15.96:0/602099130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 msgr2=0x7fd75c101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 -- 172.21.15.96:0/602099130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 msgr2=0x7fd75c100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 0x7fd75c100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 -- 172.21.15.96:0/602099130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c1010a0 con 0x7fd75c104c50 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd75b7fe640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd752ffd640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd74402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744041070 con 0x7fd75c104c50 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a1c20 con 0x7fd75c104c50 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd744004450 con 0x7fd75c104c50 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd744034440 con 0x7fd75c104c50 2024-03-18T15:40:06.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.565+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a2170 con 0x7fd75c104c50 2024-03-18T15:40:06.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.566+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c0fd780 con 0x7fd75c104c50 2024-03-18T15:40:06.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.567+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fd744044070 con 0x7fd75c104c50 2024-03-18T15:40:06.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.568+0000 7fd7597fa640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 0x7fd7480a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.568+0000 7fd75b7fe640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 0x7fd7480a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.568+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fd7440f05c0 con 0x7fd75c104c50 2024-03-18T15:40:06.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.568+0000 7fd75b7fe640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 0x7fd7480a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009fc0 tx=0x7fd74c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.572+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd7440bd020 con 0x7fd75c104c50 2024-03-18T15:40:06.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.784+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd75c02d1b0 con 0x7fd75c104c50 2024-03-18T15:40:06.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.784+0000 7fd7597fa640 1 -- 172.21.15.96:0/602099130 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7fd7440c10c0 con 0x7fd75c104c50 2024-03-18T15:40:06.785 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:06.785 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:40:03.325244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:40:06.786 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-03-18T15:40:06.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 msgr2=0x7fd7480a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 0x7fd7480a4d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009fc0 tx=0x7fd74c005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 msgr2=0x7fd75c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd744000c00 tx=0x7fd74402c640 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 shutdown_connections 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd7480a2890 0x7fd7480a4d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd75c107580 0x7fd75c100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd75c104c50 0x7fd75c1002d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 --2- 172.21.15.96:0/602099130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd75c102320 0x7fd75c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.786+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 >> 172.21.15.96:0/602099130 conn(0x7fd75c0fbc90 msgr2=0x7fd75c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.787+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 shutdown_connections 2024-03-18T15:40:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.787+0000 7fd761b22640 1 -- 172.21.15.96:0/602099130 wait complete. 2024-03-18T15:40:06.797 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3688969445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:06.797 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4232405653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:06.797 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:06 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/194543946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:06.798 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-18T15:40:06.799 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-18T15:40:06.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 -- 172.21.15.96:0/3692072560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c441022b0 msgr2=0x7f2c44108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/3692072560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c441022b0 0x7f2c44108b40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009ca0 tx=0x7f2c3c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 -- 172.21.15.96:0/3692072560 shutdown_connections 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/3692072560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c441022b0 0x7f2c44108b40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/3692072560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 0x7f2c44101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/3692072560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c44100700 0x7f2c44100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 -- 172.21.15.96:0/3692072560 >> 172.21.15.96:0/3692072560 conn(0x7f2c440fbc70 msgr2=0x7f2c440fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:06.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 -- 172.21.15.96:0/3692072560 shutdown_connections 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.900+0000 7f2c49a6c640 1 -- 172.21.15.96:0/3692072560 wait complete. 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 Processor -- start 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 -- start start 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 0x7f2c44198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c441022b0 0x7f2c4419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410c380 con 0x7f2c44100700 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410c200 con 0x7f2c441022b0 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c49a6c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4410c500 con 0x7f2c441018f0 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40292/0 (socket says 172.21.15.96:40292) 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 -- 172.21.15.96:0/1621415658 learned_addr learned my addr 172.21.15.96:0/1621415658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c3bfff640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 0x7f2c44198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c43fff640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c441022b0 0x7f2c4419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 -- 172.21.15.96:0/1621415658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 msgr2=0x7f2c44198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 0x7f2c44198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 -- 172.21.15.96:0/1621415658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c441022b0 msgr2=0x7f2c4419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c441022b0 0x7f2c4419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 -- 172.21.15.96:0/1621415658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4419f6f0 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c437fe640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400e9c0 tx=0x7f2c3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.901+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3400cd80 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.902+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4419f9e0 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.902+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c340035f0 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.902+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34010690 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.902+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4419ff80 con 0x7f2c44100700 2024-03-18T15:40:06.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.902+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c441a01b0 con 0x7f2c44100700 2024-03-18T15:40:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.904+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f2c340037a0 con 0x7f2c44100700 2024-03-18T15:40:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.904+0000 7f2c417fa640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 0x7f2c300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.904+0000 7f2c3bfff640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 0x7f2c300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.904+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2c340cf010 con 0x7f2c44100700 2024-03-18T15:40:06.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.905+0000 7f2c3bfff640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 0x7f2c300a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c44198fe0 tx=0x7f2c2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:06.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:06.908+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2c3409c050 con 0x7f2c44100700 2024-03-18T15:40:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3688969445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4232405653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:06 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/194543946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:07.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.079+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c441a0380 con 0x7f2c300a2890 2024-03-18T15:40:07.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.086+0000 7f2c417fa640 1 -- 172.21.15.96:0/1621415658 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259128 (secure 0 0 0) 0x7f2c441a0380 con 0x7f2c300a2890 2024-03-18T15:40:07.089 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:07.091 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 msgr2=0x7f2c300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 0x7f2c300a4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c44198fe0 tx=0x7f2c2c005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 msgr2=0x7f2c44197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400e9c0 tx=0x7f2c3400ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 shutdown_connections 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2c300a2890 0x7f2c300a4d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c441022b0 0x7f2c4419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c441018f0 0x7f2c44198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 --2- 172.21.15.96:0/1621415658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c44100700 0x7f2c44197ad0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 >> 172.21.15.96:0/1621415658 conn(0x7f2c440fbc70 msgr2=0x7f2c44068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 shutdown_connections 2024-03-18T15:40:07.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.089+0000 7f2c49a6c640 1 -- 172.21.15.96:0/1621415658 wait complete. 2024-03-18T15:40:07.101 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-03-18T15:40:05.473688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":784,"num_write":433,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365392,"kb_used_data":12040,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576432,"statfs":{"total":1151940427776,"available":1151566266368,"internally_reserved":0,"allocated":12328960,"data_stored":7643382,"data_compressed":226565,"data_compressed_allocated":307200,"data_compressed_original":2113536,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8900,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":61,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002053"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636733+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:01.636733+0000","last_peered":"2024-03-18T15:40:01.636733+0000","last_clean":"2024-03-18T15:40:01.636733+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:01.636733+0000","last_undegraded":"2024-03-18T15:40:01.636733+0000","last_fullsized":"2024-03-18T15:40:01.636733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.489926+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:39:31.489926+0000","last_peered":"2024-03-18T15:39:31.489926+0000","last_clean":"2024-03-18T15:39:31.489926+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:39:31.489926+0000","last_undegraded":"2024-03-18T15:39:31.489926+0000","last_fullsized":"2024-03-18T15:39:31.489926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.487632+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:39:31.487632+0000","last_peered":"2024-03-18T15:39:31.487632+0000","last_clean":"2024-03-18T15:39:31.487632+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:39:31.487632+0000","last_undegraded":"2024-03-18T15:39:31.487632+0000","last_fullsized":"2024-03-18T15:39:31.487632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973930+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973930+0000","last_peered":"2024-03-18T15:40:02.973930+0000","last_clean":"2024-03-18T15:40:02.973930+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973930+0000","last_undegraded":"2024-03-18T15:40:02.973930+0000","last_fullsized":"2024-03-18T15:40:02.973930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637015+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:01.637015+0000","last_peered":"2024-03-18T15:40:01.637015+0000","last_clean":"2024-03-18T15:40:01.637015+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:01.637015+0000","last_undegraded":"2024-03-18T15:40:01.637015+0000","last_fullsized":"2024-03-18T15:40:01.637015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788937+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:01.788937+0000","last_peered":"2024-03-18T15:40:01.788937+0000","last_clean":"2024-03-18T15:40:01.788937+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:01.788937+0000","last_undegraded":"2024-03-18T15:40:01.788937+0000","last_fullsized":"2024-03-18T15:40:01.788937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.479370+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:39:31.479370+0000","last_peered":"2024-03-18T15:39:31.479370+0000","last_clean":"2024-03-18T15:39:31.479370+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:39:31.479370+0000","last_undegraded":"2024-03-18T15:39:31.479370+0000","last_fullsized":"2024-03-18T15:39:31.479370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.487738+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:39:31.487738+0000","last_peered":"2024-03-18T15:39:31.487738+0000","last_clean":"2024-03-18T15:39:31.487738+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:39:31.487738+0000","last_undegraded":"2024-03-18T15:39:31.487738+0000","last_fullsized":"2024-03-18T15:39:31.487738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636620+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:01.636620+0000","last_peered":"2024-03-18T15:40:01.636620+0000","last_clean":"2024-03-18T15:40:01.636620+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:01.636620+0000","last_undegraded":"2024-03-18T15:40:01.636620+0000","last_fullsized":"2024-03-18T15:40:01.636620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.275148+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:03.275148+0000","last_peered":"2024-03-18T15:40:03.275148+0000","last_clean":"2024-03-18T15:40:03.275148+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:03.275148+0000","last_undegraded":"2024-03-18T15:40:03.275148+0000","last_fullsized":"2024-03-18T15:40:03.275148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637590+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:01.637590+0000","last_peered":"2024-03-18T15:40:01.637590+0000","last_clean":"2024-03-18T15:40:01.637590+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:01.637590+0000","last_undegraded":"2024-03-18T15:40:01.637590+0000","last_fullsized":"2024-03-18T15:40:01.637590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.483471+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:39:31.483471+0000","last_peered":"2024-03-18T15:39:31.483471+0000","last_clean":"2024-03-18T15:39:31.483471+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:39:31.483471+0000","last_undegraded":"2024-03-18T15:39:31.483471+0000","last_fullsized":"2024-03-18T15:39:31.483471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634707+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:02.634707+0000","last_peered":"2024-03-18T15:40:02.634707+0000","last_clean":"2024-03-18T15:40:02.634707+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:02.634707+0000","last_undegraded":"2024-03-18T15:40:02.634707+0000","last_fullsized":"2024-03-18T15:40:02.634707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788710+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:01.788710+0000","last_peered":"2024-03-18T15:40:01.788710+0000","last_clean":"2024-03-18T15:40:01.788710+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:01.788710+0000","last_undegraded":"2024-03-18T15:40:01.788710+0000","last_fullsized":"2024-03-18T15:40:01.788710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637082+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:01.637082+0000","last_peered":"2024-03-18T15:40:01.637082+0000","last_clean":"2024-03-18T15:40:01.637082+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:01.637082+0000","last_undegraded":"2024-03-18T15:40:01.637082+0000","last_fullsized":"2024-03-18T15:40:01.637082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":258,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.267786+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:05.267786+0000","last_peered":"2024-03-18T15:40:05.267786+0000","last_clean":"2024-03-18T15:40:05.267786+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:05.267786+0000","last_undegraded":"2024-03-18T15:40:05.267786+0000","last_fullsized":"2024-03-18T15:40:05.267786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.485106+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:39:31.485106+0000","last_peered":"2024-03-18T15:39:31.485106+0000","last_clean":"2024-03-18T15:39:31.485106+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:39:31.485106+0000","last_undegraded":"2024-03-18T15:39:31.485106+0000","last_fullsized":"2024-03-18T15:39:31.485106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.268965+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:05.268965+0000","last_peered":"2024-03-18T15:40:05.268965+0000","last_clean":"2024-03-18T15:40:05.268965+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:05.268965+0000","last_undegraded":"2024-03-18T15:40:05.268965+0000","last_fullsized":"2024-03-18T15:40:05.268965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":189,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2024-03-18T15:39:02.571961+0000","last_change":"2024-03-18T15:40:02.048072+0000","last_active":"2024-03-18T15:40:02.048072+0000","last_peered":"2024-03-18T15:39:02.571961+0000","last_clean":"2024-03-18T15:39:02.571961+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:39:02.571961+0000","last_undegraded":"2024-03-18T15:39:02.571961+0000","last_fullsized":"2024-03-18T15:39:02.571961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788725+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:01.788725+0000","last_peered":"2024-03-18T15:40:01.788725+0000","last_clean":"2024-03-18T15:40:01.788725+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:01.788725+0000","last_undegraded":"2024-03-18T15:40:01.788725+0000","last_fullsized":"2024-03-18T15:40:01.788725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.270266+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:03.270266+0000","last_peered":"2024-03-18T15:40:03.270266+0000","last_clean":"2024-03-18T15:40:03.270266+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:03.270266+0000","last_undegraded":"2024-03-18T15:40:03.270266+0000","last_fullsized":"2024-03-18T15:40:03.270266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636755+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:01.636755+0000","last_peered":"2024-03-18T15:40:01.636755+0000","last_clean":"2024-03-18T15:40:01.636755+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:01.636755+0000","last_undegraded":"2024-03-18T15:40:01.636755+0000","last_fullsized":"2024-03-18T15:40:01.636755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636569+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:01.636569+0000","last_peered":"2024-03-18T15:40:01.636569+0000","last_clean":"2024-03-18T15:40:01.636569+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:01.636569+0000","last_undegraded":"2024-03-18T15:40:01.636569+0000","last_fullsized":"2024-03-18T15:40:01.636569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827860+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827860+0000","last_peered":"2024-03-18T15:40:02.827860+0000","last_clean":"2024-03-18T15:40:02.827860+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827860+0000","last_undegraded":"2024-03-18T15:40:02.827860+0000","last_fullsized":"2024-03-18T15:40:02.827860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639678+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639678+0000","last_peered":"2024-03-18T15:40:02.639678+0000","last_clean":"2024-03-18T15:40:02.639678+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639678+0000","last_undegraded":"2024-03-18T15:40:02.639678+0000","last_fullsized":"2024-03-18T15:40:02.639678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639330+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:02.639330+0000","last_peered":"2024-03-18T15:40:02.639330+0000","last_clean":"2024-03-18T15:40:02.639330+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:02.639330+0000","last_undegraded":"2024-03-18T15:40:02.639330+0000","last_fullsized":"2024-03-18T15:40:02.639330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639749+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639749+0000","last_peered":"2024-03-18T15:40:02.639749+0000","last_clean":"2024-03-18T15:40:02.639749+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639749+0000","last_undegraded":"2024-03-18T15:40:02.639749+0000","last_fullsized":"2024-03-18T15:40:02.639749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634497+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634497+0000","last_peered":"2024-03-18T15:40:02.634497+0000","last_clean":"2024-03-18T15:40:02.634497+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634497+0000","last_undegraded":"2024-03-18T15:40:02.634497+0000","last_fullsized":"2024-03-18T15:40:02.634497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788655+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:01.788655+0000","last_peered":"2024-03-18T15:40:01.788655+0000","last_clean":"2024-03-18T15:40:01.788655+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:01.788655+0000","last_undegraded":"2024-03-18T15:40:01.788655+0000","last_fullsized":"2024-03-18T15:40:01.788655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637389+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:01.637389+0000","last_peered":"2024-03-18T15:40:01.637389+0000","last_clean":"2024-03-18T15:40:01.637389+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:01.637389+0000","last_undegraded":"2024-03-18T15:40:01.637389+0000","last_fullsized":"2024-03-18T15:40:01.637389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973820+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973820+0000","last_peered":"2024-03-18T15:40:02.973820+0000","last_clean":"2024-03-18T15:40:02.973820+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973820+0000","last_undegraded":"2024-03-18T15:40:02.973820+0000","last_fullsized":"2024-03-18T15:40:02.973820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788613+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:01.788613+0000","last_peered":"2024-03-18T15:40:01.788613+0000","last_clean":"2024-03-18T15:40:01.788613+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:01.788613+0000","last_undegraded":"2024-03-18T15:40:01.788613+0000","last_fullsized":"2024-03-18T15:40:01.788613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829967+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.829967+0000","last_peered":"2024-03-18T15:40:02.829967+0000","last_clean":"2024-03-18T15:40:02.829967+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.829967+0000","last_undegraded":"2024-03-18T15:40:02.829967+0000","last_fullsized":"2024-03-18T15:40:02.829967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788812+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:01.788812+0000","last_peered":"2024-03-18T15:40:01.788812+0000","last_clean":"2024-03-18T15:40:01.788812+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:01.788812+0000","last_undegraded":"2024-03-18T15:40:01.788812+0000","last_fullsized":"2024-03-18T15:40:01.788812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637466+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:01.637466+0000","last_peered":"2024-03-18T15:40:01.637466+0000","last_clean":"2024-03-18T15:40:01.637466+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:01.637466+0000","last_undegraded":"2024-03-18T15:40:01.637466+0000","last_fullsized":"2024-03-18T15:40:01.637466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.482064+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:39:31.482064+0000","last_peered":"2024-03-18T15:39:31.482064+0000","last_clean":"2024-03-18T15:39:31.482064+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:39:31.482064+0000","last_undegraded":"2024-03-18T15:39:31.482064+0000","last_fullsized":"2024-03-18T15:39:31.482064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637528+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:01.637528+0000","last_peered":"2024-03-18T15:40:01.637528+0000","last_clean":"2024-03-18T15:40:01.637528+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:01.637528+0000","last_undegraded":"2024-03-18T15:40:01.637528+0000","last_fullsized":"2024-03-18T15:40:01.637528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636192+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:01.636192+0000","last_peered":"2024-03-18T15:40:01.636192+0000","last_clean":"2024-03-18T15:40:01.636192+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:01.636192+0000","last_undegraded":"2024-03-18T15:40:01.636192+0000","last_fullsized":"2024-03-18T15:40:01.636192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639076+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639076+0000","last_peered":"2024-03-18T15:40:02.639076+0000","last_clean":"2024-03-18T15:40:02.639076+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639076+0000","last_undegraded":"2024-03-18T15:40:02.639076+0000","last_fullsized":"2024-03-18T15:40:02.639076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637634+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:01.637634+0000","last_peered":"2024-03-18T15:40:01.637634+0000","last_clean":"2024-03-18T15:40:01.637634+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:01.637634+0000","last_undegraded":"2024-03-18T15:40:01.637634+0000","last_fullsized":"2024-03-18T15:40:01.637634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639381+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:02.639381+0000","last_peered":"2024-03-18T15:40:02.639381+0000","last_clean":"2024-03-18T15:40:02.639381+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:02.639381+0000","last_undegraded":"2024-03-18T15:40:02.639381+0000","last_fullsized":"2024-03-18T15:40:02.639381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788849+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:01.788849+0000","last_peered":"2024-03-18T15:40:01.788849+0000","last_clean":"2024-03-18T15:40:01.788849+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:01.788849+0000","last_undegraded":"2024-03-18T15:40:01.788849+0000","last_fullsized":"2024-03-18T15:40:01.788849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636298+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:01.636298+0000","last_peered":"2024-03-18T15:40:01.636298+0000","last_clean":"2024-03-18T15:40:01.636298+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:01.636298+0000","last_undegraded":"2024-03-18T15:40:01.636298+0000","last_fullsized":"2024-03-18T15:40:01.636298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.484999+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:39:31.484999+0000","last_peered":"2024-03-18T15:39:31.484999+0000","last_clean":"2024-03-18T15:39:31.484999+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:39:31.484999+0000","last_undegraded":"2024-03-18T15:39:31.484999+0000","last_fullsized":"2024-03-18T15:39:31.484999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788795+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:01.788795+0000","last_peered":"2024-03-18T15:40:01.788795+0000","last_clean":"2024-03-18T15:40:01.788795+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:01.788795+0000","last_undegraded":"2024-03-18T15:40:01.788795+0000","last_fullsized":"2024-03-18T15:40:01.788795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636505+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:01.636505+0000","last_peered":"2024-03-18T15:40:01.636505+0000","last_clean":"2024-03-18T15:40:01.636505+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:01.636505+0000","last_undegraded":"2024-03-18T15:40:01.636505+0000","last_fullsized":"2024-03-18T15:40:01.636505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637423+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:01.637423+0000","last_peered":"2024-03-18T15:40:01.637423+0000","last_clean":"2024-03-18T15:40:01.637423+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:01.637423+0000","last_undegraded":"2024-03-18T15:40:01.637423+0000","last_fullsized":"2024-03-18T15:40:01.637423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637094+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:01.637094+0000","last_peered":"2024-03-18T15:40:01.637094+0000","last_clean":"2024-03-18T15:40:01.637094+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:01.637094+0000","last_undegraded":"2024-03-18T15:40:01.637094+0000","last_fullsized":"2024-03-18T15:40:01.637094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788850+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:01.788850+0000","last_peered":"2024-03-18T15:40:01.788850+0000","last_clean":"2024-03-18T15:40:01.788850+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:01.788850+0000","last_undegraded":"2024-03-18T15:40:01.788850+0000","last_fullsized":"2024-03-18T15:40:01.788850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636404+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:01.636404+0000","last_peered":"2024-03-18T15:40:01.636404+0000","last_clean":"2024-03-18T15:40:01.636404+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:01.636404+0000","last_undegraded":"2024-03-18T15:40:01.636404+0000","last_fullsized":"2024-03-18T15:40:01.636404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":675,"num_write":250,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":221499,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":461450,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469581,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":462414,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":532009,"data_compressed":10649,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":469406,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469750,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":1062047,"data_compressed":44378,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63507,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:07.105 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3688969445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4232405653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:06 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/194543946' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 -- 172.21.15.96:0/3353553029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 msgr2=0x7f43fc108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 --2- 172.21.15.96:0/3353553029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc108b40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009ca0 tx=0x7f43f402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 -- 172.21.15.96:0/3353553029 shutdown_connections 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 --2- 172.21.15.96:0/3353553029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc108b40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 --2- 172.21.15.96:0/3353553029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc1018f0 0x7f43fc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 --2- 172.21.15.96:0/3353553029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc100700 0x7f43fc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 -- 172.21.15.96:0/3353553029 >> 172.21.15.96:0/3353553029 conn(0x7f43fc0fbc70 msgr2=0x7f43fc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 -- 172.21.15.96:0/3353553029 shutdown_connections 2024-03-18T15:40:07.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.242+0000 7f4400adc640 1 -- 172.21.15.96:0/3353553029 wait complete. 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 Processor -- start 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 -- start start 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc100700 0x7f43fc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 0x7f43fc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10c380 con 0x7f43fc1022b0 2024-03-18T15:40:07.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10c200 con 0x7f43fc100700 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f4400adc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10c500 con 0x7f43fc1018f0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fa59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc100700 0x7f43fc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43f9d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 0x7f43fc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40316/0 (socket says 172.21.15.96:40316) 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 -- 172.21.15.96:0/949279421 learned_addr learned my addr 172.21.15.96:0/949279421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 -- 172.21.15.96:0/949279421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 msgr2=0x7f43fc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 0x7f43fc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 -- 172.21.15.96:0/949279421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc100700 msgr2=0x7f43fc197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc100700 0x7f43fc197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43fad9d640 1 -- 172.21.15.96:0/949279421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc19f6b0 con 0x7f43fc1022b0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.243+0000 7f43f9d9b640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 0x7f43fc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f43fad9d640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43f4037040 tx=0x7f43f4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4041020 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f4034cd0 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc19f940 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4039040 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.244+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc19fe30 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.245+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc1a0060 con 0x7f43fc1022b0 2024-03-18T15:40:07.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.246+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f43f4044020 con 0x7f43fc1022b0 2024-03-18T15:40:07.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.246+0000 7f43eb7fe640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 0x7f43e40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.246+0000 7f43fa59c640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 0x7f43e40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.246+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f43f40f0930 con 0x7f43fc1022b0 2024-03-18T15:40:07.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.247+0000 7f43fa59c640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 0x7f43e40a4d50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43fc101750 tx=0x7f43ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.251+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f43f404b020 con 0x7f43fc1022b0 2024-03-18T15:40:07.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.427+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43fc1a0230 con 0x7f43e40a2890 2024-03-18T15:40:07.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.434+0000 7f43eb7fe640 1 -- 172.21.15.96:0/949279421 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259128 (secure 0 0 0) 0x7f43fc1a0230 con 0x7f43e40a2890 2024-03-18T15:40:07.437 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:07.439 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:07.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 msgr2=0x7f43e40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 0x7f43e40a4d50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f43fc101750 tx=0x7f43ec005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:07.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 msgr2=0x7f43fc19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43f4037040 tx=0x7f43f4032f40 comp rx=0 tx=0).stop 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 shutdown_connections 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43e40a2890 0x7f43e40a4d50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43fc1022b0 0x7f43fc19f040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc1018f0 0x7f43fc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 --2- 172.21.15.96:0/949279421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43fc100700 0x7f43fc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 >> 172.21.15.96:0/949279421 conn(0x7f43fc0fbc70 msgr2=0x7f43fc0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 shutdown_connections 2024-03-18T15:40:07.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.437+0000 7f4400adc640 1 -- 172.21.15.96:0/949279421 wait complete. 2024-03-18T15:40:07.448 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-03-18T15:40:05.473688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":784,"num_write":433,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365392,"kb_used_data":12040,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576432,"statfs":{"total":1151940427776,"available":1151566266368,"internally_reserved":0,"allocated":12328960,"data_stored":7643382,"data_compressed":226565,"data_compressed_allocated":307200,"data_compressed_original":2113536,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8900,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":61,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002053"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636733+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:01.636733+0000","last_peered":"2024-03-18T15:40:01.636733+0000","last_clean":"2024-03-18T15:40:01.636733+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:01.636733+0000","last_undegraded":"2024-03-18T15:40:01.636733+0000","last_fullsized":"2024-03-18T15:40:01.636733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.489926+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:39:31.489926+0000","last_peered":"2024-03-18T15:39:31.489926+0000","last_clean":"2024-03-18T15:39:31.489926+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:39:31.489926+0000","last_undegraded":"2024-03-18T15:39:31.489926+0000","last_fullsized":"2024-03-18T15:39:31.489926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.487632+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:39:31.487632+0000","last_peered":"2024-03-18T15:39:31.487632+0000","last_clean":"2024-03-18T15:39:31.487632+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:39:31.487632+0000","last_undegraded":"2024-03-18T15:39:31.487632+0000","last_fullsized":"2024-03-18T15:39:31.487632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973930+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973930+0000","last_peered":"2024-03-18T15:40:02.973930+0000","last_clean":"2024-03-18T15:40:02.973930+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973930+0000","last_undegraded":"2024-03-18T15:40:02.973930+0000","last_fullsized":"2024-03-18T15:40:02.973930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637015+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:01.637015+0000","last_peered":"2024-03-18T15:40:01.637015+0000","last_clean":"2024-03-18T15:40:01.637015+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:01.637015+0000","last_undegraded":"2024-03-18T15:40:01.637015+0000","last_fullsized":"2024-03-18T15:40:01.637015+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788937+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:01.788937+0000","last_peered":"2024-03-18T15:40:01.788937+0000","last_clean":"2024-03-18T15:40:01.788937+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:01.788937+0000","last_undegraded":"2024-03-18T15:40:01.788937+0000","last_fullsized":"2024-03-18T15:40:01.788937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.479370+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:39:31.479370+0000","last_peered":"2024-03-18T15:39:31.479370+0000","last_clean":"2024-03-18T15:39:31.479370+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:39:31.479370+0000","last_undegraded":"2024-03-18T15:39:31.479370+0000","last_fullsized":"2024-03-18T15:39:31.479370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.487738+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:39:31.487738+0000","last_peered":"2024-03-18T15:39:31.487738+0000","last_clean":"2024-03-18T15:39:31.487738+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:39:31.487738+0000","last_undegraded":"2024-03-18T15:39:31.487738+0000","last_fullsized":"2024-03-18T15:39:31.487738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636620+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:01.636620+0000","last_peered":"2024-03-18T15:40:01.636620+0000","last_clean":"2024-03-18T15:40:01.636620+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:01.636620+0000","last_undegraded":"2024-03-18T15:40:01.636620+0000","last_fullsized":"2024-03-18T15:40:01.636620+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.275148+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:03.275148+0000","last_peered":"2024-03-18T15:40:03.275148+0000","last_clean":"2024-03-18T15:40:03.275148+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:03.275148+0000","last_undegraded":"2024-03-18T15:40:03.275148+0000","last_fullsized":"2024-03-18T15:40:03.275148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637590+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:01.637590+0000","last_peered":"2024-03-18T15:40:01.637590+0000","last_clean":"2024-03-18T15:40:01.637590+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:01.637590+0000","last_undegraded":"2024-03-18T15:40:01.637590+0000","last_fullsized":"2024-03-18T15:40:01.637590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.483471+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:39:31.483471+0000","last_peered":"2024-03-18T15:39:31.483471+0000","last_clean":"2024-03-18T15:39:31.483471+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:39:31.483471+0000","last_undegraded":"2024-03-18T15:39:31.483471+0000","last_fullsized":"2024-03-18T15:39:31.483471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634707+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:02.634707+0000","last_peered":"2024-03-18T15:40:02.634707+0000","last_clean":"2024-03-18T15:40:02.634707+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:02.634707+0000","last_undegraded":"2024-03-18T15:40:02.634707+0000","last_fullsized":"2024-03-18T15:40:02.634707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788710+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:01.788710+0000","last_peered":"2024-03-18T15:40:01.788710+0000","last_clean":"2024-03-18T15:40:01.788710+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:01.788710+0000","last_undegraded":"2024-03-18T15:40:01.788710+0000","last_fullsized":"2024-03-18T15:40:01.788710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637082+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:01.637082+0000","last_peered":"2024-03-18T15:40:01.637082+0000","last_clean":"2024-03-18T15:40:01.637082+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:01.637082+0000","last_undegraded":"2024-03-18T15:40:01.637082+0000","last_fullsized":"2024-03-18T15:40:01.637082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":258,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.267786+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:05.267786+0000","last_peered":"2024-03-18T15:40:05.267786+0000","last_clean":"2024-03-18T15:40:05.267786+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:05.267786+0000","last_undegraded":"2024-03-18T15:40:05.267786+0000","last_fullsized":"2024-03-18T15:40:05.267786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.485106+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:39:31.485106+0000","last_peered":"2024-03-18T15:39:31.485106+0000","last_clean":"2024-03-18T15:39:31.485106+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:39:31.485106+0000","last_undegraded":"2024-03-18T15:39:31.485106+0000","last_fullsized":"2024-03-18T15:39:31.485106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.268965+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:05.268965+0000","last_peered":"2024-03-18T15:40:05.268965+0000","last_clean":"2024-03-18T15:40:05.268965+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:05.268965+0000","last_undegraded":"2024-03-18T15:40:05.268965+0000","last_fullsized":"2024-03-18T15:40:05.268965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":189,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":2,"reported_epoch":89,"state":"peering","last_fresh":"2024-03-18T15:39:02.571961+0000","last_change":"2024-03-18T15:40:02.048072+0000","last_active":"2024-03-18T15:40:02.048072+0000","last_peered":"2024-03-18T15:39:02.571961+0000","last_clean":"2024-03-18T15:39:02.571961+0000","last_became_active":"2024-03-18T15:38:03.834230+0000","last_became_peered":"2024-03-18T15:38:03.834230+0000","last_unstale":"2024-03-18T15:39:02.571961+0000","last_undegraded":"2024-03-18T15:39:02.571961+0000","last_fullsized":"2024-03-18T15:39:02.571961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788725+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:01.788725+0000","last_peered":"2024-03-18T15:40:01.788725+0000","last_clean":"2024-03-18T15:40:01.788725+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:01.788725+0000","last_undegraded":"2024-03-18T15:40:01.788725+0000","last_fullsized":"2024-03-18T15:40:01.788725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.270266+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:03.270266+0000","last_peered":"2024-03-18T15:40:03.270266+0000","last_clean":"2024-03-18T15:40:03.270266+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:03.270266+0000","last_undegraded":"2024-03-18T15:40:03.270266+0000","last_fullsized":"2024-03-18T15:40:03.270266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636755+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:01.636755+0000","last_peered":"2024-03-18T15:40:01.636755+0000","last_clean":"2024-03-18T15:40:01.636755+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:01.636755+0000","last_undegraded":"2024-03-18T15:40:01.636755+0000","last_fullsized":"2024-03-18T15:40:01.636755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636569+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:01.636569+0000","last_peered":"2024-03-18T15:40:01.636569+0000","last_clean":"2024-03-18T15:40:01.636569+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:01.636569+0000","last_undegraded":"2024-03-18T15:40:01.636569+0000","last_fullsized":"2024-03-18T15:40:01.636569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827860+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827860+0000","last_peered":"2024-03-18T15:40:02.827860+0000","last_clean":"2024-03-18T15:40:02.827860+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827860+0000","last_undegraded":"2024-03-18T15:40:02.827860+0000","last_fullsized":"2024-03-18T15:40:02.827860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639678+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639678+0000","last_peered":"2024-03-18T15:40:02.639678+0000","last_clean":"2024-03-18T15:40:02.639678+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639678+0000","last_undegraded":"2024-03-18T15:40:02.639678+0000","last_fullsized":"2024-03-18T15:40:02.639678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639330+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:02.639330+0000","last_peered":"2024-03-18T15:40:02.639330+0000","last_clean":"2024-03-18T15:40:02.639330+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:02.639330+0000","last_undegraded":"2024-03-18T15:40:02.639330+0000","last_fullsized":"2024-03-18T15:40:02.639330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639749+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639749+0000","last_peered":"2024-03-18T15:40:02.639749+0000","last_clean":"2024-03-18T15:40:02.639749+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639749+0000","last_undegraded":"2024-03-18T15:40:02.639749+0000","last_fullsized":"2024-03-18T15:40:02.639749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634497+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634497+0000","last_peered":"2024-03-18T15:40:02.634497+0000","last_clean":"2024-03-18T15:40:02.634497+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634497+0000","last_undegraded":"2024-03-18T15:40:02.634497+0000","last_fullsized":"2024-03-18T15:40:02.634497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788655+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:01.788655+0000","last_peered":"2024-03-18T15:40:01.788655+0000","last_clean":"2024-03-18T15:40:01.788655+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:01.788655+0000","last_undegraded":"2024-03-18T15:40:01.788655+0000","last_fullsized":"2024-03-18T15:40:01.788655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637389+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:01.637389+0000","last_peered":"2024-03-18T15:40:01.637389+0000","last_clean":"2024-03-18T15:40:01.637389+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:01.637389+0000","last_undegraded":"2024-03-18T15:40:01.637389+0000","last_fullsized":"2024-03-18T15:40:01.637389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973820+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973820+0000","last_peered":"2024-03-18T15:40:02.973820+0000","last_clean":"2024-03-18T15:40:02.973820+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973820+0000","last_undegraded":"2024-03-18T15:40:02.973820+0000","last_fullsized":"2024-03-18T15:40:02.973820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788613+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:01.788613+0000","last_peered":"2024-03-18T15:40:01.788613+0000","last_clean":"2024-03-18T15:40:01.788613+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:01.788613+0000","last_undegraded":"2024-03-18T15:40:01.788613+0000","last_fullsized":"2024-03-18T15:40:01.788613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829967+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.829967+0000","last_peered":"2024-03-18T15:40:02.829967+0000","last_clean":"2024-03-18T15:40:02.829967+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.829967+0000","last_undegraded":"2024-03-18T15:40:02.829967+0000","last_fullsized":"2024-03-18T15:40:02.829967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788812+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:01.788812+0000","last_peered":"2024-03-18T15:40:01.788812+0000","last_clean":"2024-03-18T15:40:01.788812+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:01.788812+0000","last_undegraded":"2024-03-18T15:40:01.788812+0000","last_fullsized":"2024-03-18T15:40:01.788812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637466+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:01.637466+0000","last_peered":"2024-03-18T15:40:01.637466+0000","last_clean":"2024-03-18T15:40:01.637466+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:01.637466+0000","last_undegraded":"2024-03-18T15:40:01.637466+0000","last_fullsized":"2024-03-18T15:40:01.637466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.482064+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:39:31.482064+0000","last_peered":"2024-03-18T15:39:31.482064+0000","last_clean":"2024-03-18T15:39:31.482064+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:39:31.482064+0000","last_undegraded":"2024-03-18T15:39:31.482064+0000","last_fullsized":"2024-03-18T15:39:31.482064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637528+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:01.637528+0000","last_peered":"2024-03-18T15:40:01.637528+0000","last_clean":"2024-03-18T15:40:01.637528+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:01.637528+0000","last_undegraded":"2024-03-18T15:40:01.637528+0000","last_fullsized":"2024-03-18T15:40:01.637528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636192+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:01.636192+0000","last_peered":"2024-03-18T15:40:01.636192+0000","last_clean":"2024-03-18T15:40:01.636192+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:01.636192+0000","last_undegraded":"2024-03-18T15:40:01.636192+0000","last_fullsized":"2024-03-18T15:40:01.636192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639076+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639076+0000","last_peered":"2024-03-18T15:40:02.639076+0000","last_clean":"2024-03-18T15:40:02.639076+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639076+0000","last_undegraded":"2024-03-18T15:40:02.639076+0000","last_fullsized":"2024-03-18T15:40:02.639076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637634+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:01.637634+0000","last_peered":"2024-03-18T15:40:01.637634+0000","last_clean":"2024-03-18T15:40:01.637634+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:01.637634+0000","last_undegraded":"2024-03-18T15:40:01.637634+0000","last_fullsized":"2024-03-18T15:40:01.637634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639381+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:02.639381+0000","last_peered":"2024-03-18T15:40:02.639381+0000","last_clean":"2024-03-18T15:40:02.639381+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:02.639381+0000","last_undegraded":"2024-03-18T15:40:02.639381+0000","last_fullsized":"2024-03-18T15:40:02.639381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788849+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:01.788849+0000","last_peered":"2024-03-18T15:40:01.788849+0000","last_clean":"2024-03-18T15:40:01.788849+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:01.788849+0000","last_undegraded":"2024-03-18T15:40:01.788849+0000","last_fullsized":"2024-03-18T15:40:01.788849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636298+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:01.636298+0000","last_peered":"2024-03-18T15:40:01.636298+0000","last_clean":"2024-03-18T15:40:01.636298+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:01.636298+0000","last_undegraded":"2024-03-18T15:40:01.636298+0000","last_fullsized":"2024-03-18T15:40:01.636298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-18T15:39:31.484999+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:39:31.484999+0000","last_peered":"2024-03-18T15:39:31.484999+0000","last_clean":"2024-03-18T15:39:31.484999+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:39:31.484999+0000","last_undegraded":"2024-03-18T15:39:31.484999+0000","last_fullsized":"2024-03-18T15:39:31.484999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788795+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:01.788795+0000","last_peered":"2024-03-18T15:40:01.788795+0000","last_clean":"2024-03-18T15:40:01.788795+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:01.788795+0000","last_undegraded":"2024-03-18T15:40:01.788795+0000","last_fullsized":"2024-03-18T15:40:01.788795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636505+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:01.636505+0000","last_peered":"2024-03-18T15:40:01.636505+0000","last_clean":"2024-03-18T15:40:01.636505+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:01.636505+0000","last_undegraded":"2024-03-18T15:40:01.636505+0000","last_fullsized":"2024-03-18T15:40:01.636505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637423+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:01.637423+0000","last_peered":"2024-03-18T15:40:01.637423+0000","last_clean":"2024-03-18T15:40:01.637423+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:01.637423+0000","last_undegraded":"2024-03-18T15:40:01.637423+0000","last_fullsized":"2024-03-18T15:40:01.637423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.637094+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:01.637094+0000","last_peered":"2024-03-18T15:40:01.637094+0000","last_clean":"2024-03-18T15:40:01.637094+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:01.637094+0000","last_undegraded":"2024-03-18T15:40:01.637094+0000","last_fullsized":"2024-03-18T15:40:01.637094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788850+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:01.788850+0000","last_peered":"2024-03-18T15:40:01.788850+0000","last_clean":"2024-03-18T15:40:01.788850+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:01.788850+0000","last_undegraded":"2024-03-18T15:40:01.788850+0000","last_fullsized":"2024-03-18T15:40:01.788850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.636404+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:01.636404+0000","last_peered":"2024-03-18T15:40:01.636404+0000","last_clean":"2024-03-18T15:40:01.636404+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:01.636404+0000","last_undegraded":"2024-03-18T15:40:01.636404+0000","last_fullsized":"2024-03-18T15:40:01.636404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":675,"num_write":250,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":221499,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":461450,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469581,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":462414,"data_compressed":5855,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":532009,"data_compressed":10649,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":469406,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469750,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":1062047,"data_compressed":44378,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63507,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:07.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:07.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 -- 172.21.15.96:0/585983184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec1022c0 msgr2=0x7fb1ec108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/585983184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec108b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009ca0 tx=0x7fb1e402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 -- 172.21.15.96:0/585983184 shutdown_connections 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/585983184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec108b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/585983184 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec101900 0x7fb1ec101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/585983184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec100710 0x7fb1ec100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 -- 172.21.15.96:0/585983184 >> 172.21.15.96:0/585983184 conn(0x7fb1ec0fbcc0 msgr2=0x7fb1ec0fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 -- 172.21.15.96:0/585983184 shutdown_connections 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.589+0000 7fb1f1edc640 1 -- 172.21.15.96:0/585983184 wait complete. 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 Processor -- start 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 -- start start 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec100710 0x7fb1ec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10d4b0 con 0x7fb1ec101900 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10d330 con 0x7fb1ec100710 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1f1edc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec10d630 con 0x7fb1ec1022c0 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40340/0 (socket says 172.21.15.96:40340) 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 -- 172.21.15.96:0/957414039 learned_addr learned my addr 172.21.15.96:0/957414039 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1ebfff640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1eb7fe640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec100710 0x7fb1ec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 -- 172.21.15.96:0/957414039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec1022c0 msgr2=0x7fb1ec19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 -- 172.21.15.96:0/957414039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec100710 msgr2=0x7fb1ec197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.590+0000 7fb1e3fff640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec100710 0x7fb1ec197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1e3fff640 1 -- 172.21.15.96:0/957414039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec19f6d0 con 0x7fb1ec101900 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1e3fff640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400cd80 con 0x7fb1ec101900 2024-03-18T15:40:07.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d4010910 con 0x7fb1ec101900 2024-03-18T15:40:07.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec19f9c0 con 0x7fb1ec101900 2024-03-18T15:40:07.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4018990 con 0x7fb1ec101900 2024-03-18T15:40:07.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.591+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec19ff10 con 0x7fb1ec101900 2024-03-18T15:40:07.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.592+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1ec1079d0 con 0x7fb1ec101900 2024-03-18T15:40:07.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.593+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fb1d4010ac0 con 0x7fb1ec101900 2024-03-18T15:40:07.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.594+0000 7fb1e97fa640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 0x7fb1d80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.594+0000 7fb1eb7fe640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 0x7fb1d80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.594+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb1d40cf340 con 0x7fb1ec101900 2024-03-18T15:40:07.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.594+0000 7fb1eb7fe640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 0x7fb1d80a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec101760 tx=0x7fb1dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.597+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb1d409bc00 con 0x7fb1ec101900 2024-03-18T15:40:07.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.770+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1ec061c20 con 0x7fb1d80a2890 2024-03-18T15:40:07.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.777+0000 7fb1e97fa640 1 -- 172.21.15.96:0/957414039 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259185 (secure 0 0 0) 0x7fb1ec061c20 con 0x7fb1d80a2890 2024-03-18T15:40:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:07.782 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 msgr2=0x7fb1d80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 0x7fb1d80a4d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec101760 tx=0x7fb1dc005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 msgr2=0x7fb1ec197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400e9f0 tx=0x7fb1d400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 shutdown_connections 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1d80a2890 0x7fb1d80a4d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1ec1022c0 0x7fb1ec19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1ec101900 0x7fb1ec197ff0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 --2- 172.21.15.96:0/957414039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1ec100710 0x7fb1ec197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 >> 172.21.15.96:0/957414039 conn(0x7fb1ec0fbcc0 msgr2=0x7fb1ec104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 shutdown_connections 2024-03-18T15:40:07.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.780+0000 7fb1f1edc640 1 -- 172.21.15.96:0/957414039 wait complete. 2024-03-18T15:40:07.791 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:07 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/602099130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:07.791 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:07 smithi096 ceph-mon[31930]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:07.791 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:07 smithi096 ceph-mon[31930]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:07.791 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:07 smithi096 ceph-mon[31930]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:07.792 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-03-18T15:40:07.474220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":784,"num_write":433,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365472,"kb_used_data":12108,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576352,"statfs":{"total":1151940427776,"available":1151566184448,"internally_reserved":0,"allocated":12398592,"data_stored":7686699,"data_compressed":241207,"data_compressed_allocated":327680,"data_compressed_original":2154496,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8900,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":56,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002104"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637350+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:02.637350+0000","last_peered":"2024-03-18T15:40:02.637350+0000","last_clean":"2024-03-18T15:40:02.637350+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:02.637350+0000","last_undegraded":"2024-03-18T15:40:02.637350+0000","last_fullsized":"2024-03-18T15:40:02.637350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770801+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:40:02.770801+0000","last_peered":"2024-03-18T15:40:02.770801+0000","last_clean":"2024-03-18T15:40:02.770801+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:40:02.770801+0000","last_undegraded":"2024-03-18T15:40:02.770801+0000","last_fullsized":"2024-03-18T15:40:02.770801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770565+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:40:02.770565+0000","last_peered":"2024-03-18T15:40:02.770565+0000","last_clean":"2024-03-18T15:40:02.770565+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:40:02.770565+0000","last_undegraded":"2024-03-18T15:40:02.770565+0000","last_fullsized":"2024-03-18T15:40:02.770565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973930+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973930+0000","last_peered":"2024-03-18T15:40:02.973930+0000","last_clean":"2024-03-18T15:40:02.973930+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973930+0000","last_undegraded":"2024-03-18T15:40:02.973930+0000","last_fullsized":"2024-03-18T15:40:02.973930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638688+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:02.638688+0000","last_peered":"2024-03-18T15:40:02.638688+0000","last_clean":"2024-03-18T15:40:02.638688+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:02.638688+0000","last_undegraded":"2024-03-18T15:40:02.638688+0000","last_fullsized":"2024-03-18T15:40:02.638688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788937+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:01.788937+0000","last_peered":"2024-03-18T15:40:01.788937+0000","last_clean":"2024-03-18T15:40:01.788937+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:01.788937+0000","last_undegraded":"2024-03-18T15:40:01.788937+0000","last_fullsized":"2024-03-18T15:40:01.788937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770536+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:40:02.770536+0000","last_peered":"2024-03-18T15:40:02.770536+0000","last_clean":"2024-03-18T15:40:02.770536+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:40:02.770536+0000","last_undegraded":"2024-03-18T15:40:02.770536+0000","last_fullsized":"2024-03-18T15:40:02.770536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770431+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:40:02.770431+0000","last_peered":"2024-03-18T15:40:02.770431+0000","last_clean":"2024-03-18T15:40:02.770431+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:40:02.770431+0000","last_undegraded":"2024-03-18T15:40:02.770431+0000","last_fullsized":"2024-03-18T15:40:02.770431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637280+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:02.637280+0000","last_peered":"2024-03-18T15:40:02.637280+0000","last_clean":"2024-03-18T15:40:02.637280+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:02.637280+0000","last_undegraded":"2024-03-18T15:40:02.637280+0000","last_fullsized":"2024-03-18T15:40:02.637280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.275148+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:03.275148+0000","last_peered":"2024-03-18T15:40:03.275148+0000","last_clean":"2024-03-18T15:40:03.275148+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:03.275148+0000","last_undegraded":"2024-03-18T15:40:03.275148+0000","last_fullsized":"2024-03-18T15:40:03.275148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639140+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:02.639140+0000","last_peered":"2024-03-18T15:40:02.639140+0000","last_clean":"2024-03-18T15:40:02.639140+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:02.639140+0000","last_undegraded":"2024-03-18T15:40:02.639140+0000","last_fullsized":"2024-03-18T15:40:02.639140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770915+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:40:02.770915+0000","last_peered":"2024-03-18T15:40:02.770915+0000","last_clean":"2024-03-18T15:40:02.770915+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:40:02.770915+0000","last_undegraded":"2024-03-18T15:40:02.770915+0000","last_fullsized":"2024-03-18T15:40:02.770915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634707+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:02.634707+0000","last_peered":"2024-03-18T15:40:02.634707+0000","last_clean":"2024-03-18T15:40:02.634707+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:02.634707+0000","last_undegraded":"2024-03-18T15:40:02.634707+0000","last_fullsized":"2024-03-18T15:40:02.634707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788710+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:01.788710+0000","last_peered":"2024-03-18T15:40:01.788710+0000","last_clean":"2024-03-18T15:40:01.788710+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:01.788710+0000","last_undegraded":"2024-03-18T15:40:01.788710+0000","last_fullsized":"2024-03-18T15:40:01.788710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638755+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:02.638755+0000","last_peered":"2024-03-18T15:40:02.638755+0000","last_clean":"2024-03-18T15:40:02.638755+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:02.638755+0000","last_undegraded":"2024-03-18T15:40:02.638755+0000","last_fullsized":"2024-03-18T15:40:02.638755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":258,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.267786+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:05.267786+0000","last_peered":"2024-03-18T15:40:05.267786+0000","last_clean":"2024-03-18T15:40:05.267786+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:05.267786+0000","last_undegraded":"2024-03-18T15:40:05.267786+0000","last_fullsized":"2024-03-18T15:40:05.267786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770819+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:40:02.770819+0000","last_peered":"2024-03-18T15:40:02.770819+0000","last_clean":"2024-03-18T15:40:02.770819+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:40:02.770819+0000","last_undegraded":"2024-03-18T15:40:02.770819+0000","last_fullsized":"2024-03-18T15:40:02.770819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.268965+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:05.268965+0000","last_peered":"2024-03-18T15:40:05.268965+0000","last_clean":"2024-03-18T15:40:05.268965+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:05.268965+0000","last_undegraded":"2024-03-18T15:40:05.268965+0000","last_fullsized":"2024-03-18T15:40:05.268965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":189,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.649987+0000","last_change":"2024-03-18T15:40:02.649987+0000","last_active":"2024-03-18T15:40:02.649987+0000","last_peered":"2024-03-18T15:40:02.649987+0000","last_clean":"2024-03-18T15:40:02.649987+0000","last_became_active":"2024-03-18T15:40:02.640566+0000","last_became_peered":"2024-03-18T15:40:02.640566+0000","last_unstale":"2024-03-18T15:40:02.649987+0000","last_undegraded":"2024-03-18T15:40:02.649987+0000","last_fullsized":"2024-03-18T15:40:02.649987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:49:31.828795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788725+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:01.788725+0000","last_peered":"2024-03-18T15:40:01.788725+0000","last_clean":"2024-03-18T15:40:01.788725+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:01.788725+0000","last_undegraded":"2024-03-18T15:40:01.788725+0000","last_fullsized":"2024-03-18T15:40:01.788725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.270266+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:03.270266+0000","last_peered":"2024-03-18T15:40:03.270266+0000","last_clean":"2024-03-18T15:40:03.270266+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:03.270266+0000","last_undegraded":"2024-03-18T15:40:03.270266+0000","last_fullsized":"2024-03-18T15:40:03.270266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637233+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:02.637233+0000","last_peered":"2024-03-18T15:40:02.637233+0000","last_clean":"2024-03-18T15:40:02.637233+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:02.637233+0000","last_undegraded":"2024-03-18T15:40:02.637233+0000","last_fullsized":"2024-03-18T15:40:02.637233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637255+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:02.637255+0000","last_peered":"2024-03-18T15:40:02.637255+0000","last_clean":"2024-03-18T15:40:02.637255+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:02.637255+0000","last_undegraded":"2024-03-18T15:40:02.637255+0000","last_fullsized":"2024-03-18T15:40:02.637255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827860+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827860+0000","last_peered":"2024-03-18T15:40:02.827860+0000","last_clean":"2024-03-18T15:40:02.827860+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827860+0000","last_undegraded":"2024-03-18T15:40:02.827860+0000","last_fullsized":"2024-03-18T15:40:02.827860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639678+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639678+0000","last_peered":"2024-03-18T15:40:02.639678+0000","last_clean":"2024-03-18T15:40:02.639678+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639678+0000","last_undegraded":"2024-03-18T15:40:02.639678+0000","last_fullsized":"2024-03-18T15:40:02.639678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639330+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:02.639330+0000","last_peered":"2024-03-18T15:40:02.639330+0000","last_clean":"2024-03-18T15:40:02.639330+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:02.639330+0000","last_undegraded":"2024-03-18T15:40:02.639330+0000","last_fullsized":"2024-03-18T15:40:02.639330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639749+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639749+0000","last_peered":"2024-03-18T15:40:02.639749+0000","last_clean":"2024-03-18T15:40:02.639749+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639749+0000","last_undegraded":"2024-03-18T15:40:02.639749+0000","last_fullsized":"2024-03-18T15:40:02.639749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634497+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634497+0000","last_peered":"2024-03-18T15:40:02.634497+0000","last_clean":"2024-03-18T15:40:02.634497+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634497+0000","last_undegraded":"2024-03-18T15:40:02.634497+0000","last_fullsized":"2024-03-18T15:40:02.634497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788655+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:01.788655+0000","last_peered":"2024-03-18T15:40:01.788655+0000","last_clean":"2024-03-18T15:40:01.788655+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:01.788655+0000","last_undegraded":"2024-03-18T15:40:01.788655+0000","last_fullsized":"2024-03-18T15:40:01.788655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638875+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:02.638875+0000","last_peered":"2024-03-18T15:40:02.638875+0000","last_clean":"2024-03-18T15:40:02.638875+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:02.638875+0000","last_undegraded":"2024-03-18T15:40:02.638875+0000","last_fullsized":"2024-03-18T15:40:02.638875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973820+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973820+0000","last_peered":"2024-03-18T15:40:02.973820+0000","last_clean":"2024-03-18T15:40:02.973820+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973820+0000","last_undegraded":"2024-03-18T15:40:02.973820+0000","last_fullsized":"2024-03-18T15:40:02.973820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788613+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:01.788613+0000","last_peered":"2024-03-18T15:40:01.788613+0000","last_clean":"2024-03-18T15:40:01.788613+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:01.788613+0000","last_undegraded":"2024-03-18T15:40:01.788613+0000","last_fullsized":"2024-03-18T15:40:01.788613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829967+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.829967+0000","last_peered":"2024-03-18T15:40:02.829967+0000","last_clean":"2024-03-18T15:40:02.829967+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.829967+0000","last_undegraded":"2024-03-18T15:40:02.829967+0000","last_fullsized":"2024-03-18T15:40:02.829967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788812+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:01.788812+0000","last_peered":"2024-03-18T15:40:01.788812+0000","last_clean":"2024-03-18T15:40:01.788812+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:01.788812+0000","last_undegraded":"2024-03-18T15:40:01.788812+0000","last_fullsized":"2024-03-18T15:40:01.788812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639056+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:02.639056+0000","last_peered":"2024-03-18T15:40:02.639056+0000","last_clean":"2024-03-18T15:40:02.639056+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:02.639056+0000","last_undegraded":"2024-03-18T15:40:02.639056+0000","last_fullsized":"2024-03-18T15:40:02.639056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770696+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:40:02.770696+0000","last_peered":"2024-03-18T15:40:02.770696+0000","last_clean":"2024-03-18T15:40:02.770696+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:40:02.770696+0000","last_undegraded":"2024-03-18T15:40:02.770696+0000","last_fullsized":"2024-03-18T15:40:02.770696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638847+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:02.638847+0000","last_peered":"2024-03-18T15:40:02.638847+0000","last_clean":"2024-03-18T15:40:02.638847+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:02.638847+0000","last_undegraded":"2024-03-18T15:40:02.638847+0000","last_fullsized":"2024-03-18T15:40:02.638847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637056+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:02.637056+0000","last_peered":"2024-03-18T15:40:02.637056+0000","last_clean":"2024-03-18T15:40:02.637056+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:02.637056+0000","last_undegraded":"2024-03-18T15:40:02.637056+0000","last_fullsized":"2024-03-18T15:40:02.637056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639076+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639076+0000","last_peered":"2024-03-18T15:40:02.639076+0000","last_clean":"2024-03-18T15:40:02.639076+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639076+0000","last_undegraded":"2024-03-18T15:40:02.639076+0000","last_fullsized":"2024-03-18T15:40:02.639076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638975+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:02.638975+0000","last_peered":"2024-03-18T15:40:02.638975+0000","last_clean":"2024-03-18T15:40:02.638975+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:02.638975+0000","last_undegraded":"2024-03-18T15:40:02.638975+0000","last_fullsized":"2024-03-18T15:40:02.638975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639381+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:02.639381+0000","last_peered":"2024-03-18T15:40:02.639381+0000","last_clean":"2024-03-18T15:40:02.639381+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:02.639381+0000","last_undegraded":"2024-03-18T15:40:02.639381+0000","last_fullsized":"2024-03-18T15:40:02.639381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788849+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:01.788849+0000","last_peered":"2024-03-18T15:40:01.788849+0000","last_clean":"2024-03-18T15:40:01.788849+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:01.788849+0000","last_undegraded":"2024-03-18T15:40:01.788849+0000","last_fullsized":"2024-03-18T15:40:01.788849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637119+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:02.637119+0000","last_peered":"2024-03-18T15:40:02.637119+0000","last_clean":"2024-03-18T15:40:02.637119+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:02.637119+0000","last_undegraded":"2024-03-18T15:40:02.637119+0000","last_fullsized":"2024-03-18T15:40:02.637119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770759+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:40:02.770759+0000","last_peered":"2024-03-18T15:40:02.770759+0000","last_clean":"2024-03-18T15:40:02.770759+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:40:02.770759+0000","last_undegraded":"2024-03-18T15:40:02.770759+0000","last_fullsized":"2024-03-18T15:40:02.770759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788795+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:01.788795+0000","last_peered":"2024-03-18T15:40:01.788795+0000","last_clean":"2024-03-18T15:40:01.788795+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:01.788795+0000","last_undegraded":"2024-03-18T15:40:01.788795+0000","last_fullsized":"2024-03-18T15:40:01.788795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637235+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:02.637235+0000","last_peered":"2024-03-18T15:40:02.637235+0000","last_clean":"2024-03-18T15:40:02.637235+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:02.637235+0000","last_undegraded":"2024-03-18T15:40:02.637235+0000","last_fullsized":"2024-03-18T15:40:02.637235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639014+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:02.639014+0000","last_peered":"2024-03-18T15:40:02.639014+0000","last_clean":"2024-03-18T15:40:02.639014+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:02.639014+0000","last_undegraded":"2024-03-18T15:40:02.639014+0000","last_fullsized":"2024-03-18T15:40:02.639014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638711+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:02.638711+0000","last_peered":"2024-03-18T15:40:02.638711+0000","last_clean":"2024-03-18T15:40:02.638711+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:02.638711+0000","last_undegraded":"2024-03-18T15:40:02.638711+0000","last_fullsized":"2024-03-18T15:40:02.638711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788850+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:01.788850+0000","last_peered":"2024-03-18T15:40:01.788850+0000","last_clean":"2024-03-18T15:40:01.788850+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:01.788850+0000","last_undegraded":"2024-03-18T15:40:01.788850+0000","last_fullsized":"2024-03-18T15:40:01.788850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637251+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:02.637251+0000","last_peered":"2024-03-18T15:40:02.637251+0000","last_clean":"2024-03-18T15:40:02.637251+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:02.637251+0000","last_undegraded":"2024-03-18T15:40:02.637251+0000","last_fullsized":"2024-03-18T15:40:02.637251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":675,"num_write":250,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":222349,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":478900,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469581,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":479796,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":532009,"data_compressed":10649,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":477891,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469750,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":1062047,"data_compressed":44378,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63507,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:07.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-18T15:40:07.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:07 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/602099130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:07 smithi193 ceph-mon[36702]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:07 smithi193 ceph-mon[36702]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:07.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:07 smithi193 ceph-mon[36702]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 -- 172.21.15.96:0/1472737568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c1033c0 msgr2=0x7f522c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 --2- 172.21.15.96:0/1472737568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c1033c0 0x7f522c109c50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5224009ca0 tx=0x7f522402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 -- 172.21.15.96:0/1472737568 shutdown_connections 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 --2- 172.21.15.96:0/1472737568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c1033c0 0x7f522c109c50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 --2- 172.21.15.96:0/1472737568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c102a00 0x7f522c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 --2- 172.21.15.96:0/1472737568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 0x7f522c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 -- 172.21.15.96:0/1472737568 >> 172.21.15.96:0/1472737568 conn(0x7f522c0fcd80 msgr2=0x7f522c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 -- 172.21.15.96:0/1472737568 shutdown_connections 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.932+0000 7f5232e4d640 1 -- 172.21.15.96:0/1472737568 wait complete. 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 Processor -- start 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 -- start start 2024-03-18T15:40:07.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 0x7f522c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c1033c0 0x7f522c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d490 con 0x7f522c102a00 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d310 con 0x7f522c1033c0 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5232e4d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f522c10d610 con 0x7f522c101810 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40362/0 (socket says 172.21.15.96:40362) 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 -- 172.21.15.96:0/666908972 learned_addr learned my addr 172.21.15.96:0/666908972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5230be9640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 0x7f522c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f52313ea640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c1033c0 0x7f522c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 -- 172.21.15.96:0/666908972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 msgr2=0x7f522c197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 0x7f522c197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 -- 172.21.15.96:0/666908972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c1033c0 msgr2=0x7f522c19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c1033c0 0x7f522c19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 -- 172.21.15.96:0/666908972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c19f6c0 con 0x7f522c102a00 2024-03-18T15:40:07.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f522bfff640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f521800e9f0 tx=0x7f521800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521800cd80 con 0x7f522c102a00 2024-03-18T15:40:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5218010910 con 0x7f522c102a00 2024-03-18T15:40:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.933+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218018990 con 0x7f522c102a00 2024-03-18T15:40:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.934+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c19f9b0 con 0x7f522c102a00 2024-03-18T15:40:07.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.934+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c19ff00 con 0x7f522c102a00 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.935+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c108b30 con 0x7f522c102a00 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.936+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f5218010ac0 con 0x7f522c102a00 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.936+0000 7f5229ffb640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 0x7f521c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.936+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f52180cf330 con 0x7f522c102a00 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.936+0000 7f5230be9640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 0x7f521c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:07.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.937+0000 7f5230be9640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 0x7f521c0a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f522c102860 tx=0x7f5220005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:07.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:07.940+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f521809bbf0 con 0x7f522c102a00 2024-03-18T15:40:08.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:07 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/602099130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:08.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:07 smithi137 ceph-mon[35868]: from='client.15426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:08.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:07 smithi137 ceph-mon[35868]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:08.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:07 smithi137 ceph-mon[35868]: pgmap v353: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:08.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.187+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f522c02d370 con 0x7f522c102a00 2024-03-18T15:40:08.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.188+0000 7f5229ffb640 1 -- 172.21.15.96:0/666908972 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1966 (secure 0 0 0) 0x7f521809fc90 con 0x7f522c102a00 2024-03-18T15:40:08.189 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:08.189 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":488,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":90,"num_osds":12,"num_up_osds":12,"osd_up_since":1710776231,"num_in_osds":12,"osd_in_since":1710776211,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":667219,"bytes_used":374243328,"bytes_avail":1151566184448,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":8,"recovering_keys_per_sec":1,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"read_bytes_sec":7166,"write_bytes_sec":1919,"read_op_per_sec":9,"write_op_per_sec":1},"fsmap":{"epoch":25,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14628},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14640},{"filesystem_id":1,"rank":2,"name":"k","status":"up:standby-replay","gid":24392},{"filesystem_id":1,"rank":1,"name":"l","status":"up:standby-replay","gid":24415},{"filesystem_id":1,"rank":0,"name":"e","status":"up:standby-replay","gid":24422},{"filesystem_id":1,"rank":4,"name":"h","status":"up:active","gid":24452},{"filesystem_id":1,"rank":3,"name":"f","status":"up:active","gid":24460},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24472},{"filesystem_id":1,"rank":1,"name":"b","status":"up:active","gid":24485},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24493}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.96:8443/"}},"servicemap":{"epoch":25,"modified":"2024-03-18T15:39:19.461230+0000","services":{}},"progress_events":{}} 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 msgr2=0x7f521c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 0x7f521c0a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f522c102860 tx=0x7f5220005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 msgr2=0x7f522c197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f521800e9f0 tx=0x7f521800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 shutdown_connections 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f521c0a2890 0x7f521c0a4d50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f522c1033c0 0x7f522c19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f522c102a00 0x7f522c197fe0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 --2- 172.21.15.96:0/666908972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f522c101810 0x7f522c197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 >> 172.21.15.96:0/666908972 conn(0x7f522c0fcd80 msgr2=0x7f522c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 shutdown_connections 2024-03-18T15:40:08.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.190+0000 7f5232e4d640 1 -- 172.21.15.96:0/666908972 wait complete. 2024-03-18T15:40:08.202 INFO:tasks.fwd_scrub.ceph_manager:making progress, resetting timeout 2024-03-18T15:40:08.202 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- 172.21.15.96:0/1856362441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 msgr2=0x7f98a4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 --2- 172.21.15.96:0/1856362441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4100b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- 172.21.15.96:0/1856362441 shutdown_connections 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 --2- 172.21.15.96:0/1856362441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41022b0 0x7f98a4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 --2- 172.21.15.96:0/1856362441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41018f0 0x7f98a4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 --2- 172.21.15.96:0/1856362441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4100b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- 172.21.15.96:0/1856362441 >> 172.21.15.96:0/1856362441 conn(0x7f98a40fbc90 msgr2=0x7f98a40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- 172.21.15.96:0/1856362441 shutdown_connections 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- 172.21.15.96:0/1856362441 wait complete. 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 Processor -- start 2024-03-18T15:40:08.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.303+0000 7f98aa744640 1 -- start start 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41018f0 0x7f98a4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41022b0 0x7f98a419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c350 con 0x7f98a4100700 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c1d0 con 0x7f98a41022b0 2024-03-18T15:40:08.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98a410c4d0 con 0x7f98a41018f0 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40374/0 (socket says 172.21.15.96:40374) 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 -- 172.21.15.96:0/3836501316 learned_addr learned my addr 172.21.15.96:0/3836501316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a8ce1640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41022b0 0x7f98a419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a37fe640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41018f0 0x7f98a4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 -- 172.21.15.96:0/3836501316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41018f0 msgr2=0x7f98a4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41018f0 0x7f98a4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 -- 172.21.15.96:0/3836501316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41022b0 msgr2=0x7f98a419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41022b0 0x7f98a419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 -- 172.21.15.96:0/3836501316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419f770 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a3fff640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894041070 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98940044d0 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419fa00 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894034440 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.304+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a419fef0 con 0x7f98a4100700 2024-03-18T15:40:08.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.305+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a41a00c0 con 0x7f98a4100700 2024-03-18T15:40:08.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.306+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f9894044070 con 0x7f98a4100700 2024-03-18T15:40:08.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.307+0000 7f98a17fa640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 0x7f988c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:08.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.307+0000 7f98a37fe640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 0x7f988c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:08.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.307+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f98940f0600 con 0x7f98a4100700 2024-03-18T15:40:08.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.307+0000 7f98a37fe640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 0x7f988c0a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f98a4199060 tx=0x7f9898005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:08.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.311+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f98940bcfd0 con 0x7f98a4100700 2024-03-18T15:40:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.485+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98a406a540 con 0x7f988c0a2890 2024-03-18T15:40:08.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.492+0000 7f98a17fa640 1 -- 172.21.15.96:0/3836501316 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259185 (secure 0 0 0) 0x7f98a406a540 con 0x7f988c0a2890 2024-03-18T15:40:08.495 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:08.496 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:08.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 msgr2=0x7f988c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 0x7f988c0a4d50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f98a4199060 tx=0x7f9898005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 msgr2=0x7f98a4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9894000c00 tx=0x7f9894005d50 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 shutdown_connections 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f988c0a2890 0x7f988c0a4d50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98a41022b0 0x7f98a419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98a41018f0 0x7f98a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 --2- 172.21.15.96:0/3836501316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a4100700 0x7f98a4197b50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 >> 172.21.15.96:0/3836501316 conn(0x7f98a40fbc90 msgr2=0x7f98a4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 shutdown_connections 2024-03-18T15:40:08.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:08.495+0000 7f98aa744640 1 -- 172.21.15.96:0/3836501316 wait complete. 2024-03-18T15:40:08.507 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-03-18T15:40:07.474220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1303,"num_read_kb":784,"num_write":433,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365472,"kb_used_data":12108,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576352,"statfs":{"total":1151940427776,"available":1151566184448,"internally_reserved":0,"allocated":12398592,"data_stored":7686699,"data_compressed":241207,"data_compressed_allocated":327680,"data_compressed_original":2154496,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8900,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":56,"num_write":10,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002104"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637350+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:02.637350+0000","last_peered":"2024-03-18T15:40:02.637350+0000","last_clean":"2024-03-18T15:40:02.637350+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:02.637350+0000","last_undegraded":"2024-03-18T15:40:02.637350+0000","last_fullsized":"2024-03-18T15:40:02.637350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770801+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:40:02.770801+0000","last_peered":"2024-03-18T15:40:02.770801+0000","last_clean":"2024-03-18T15:40:02.770801+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:40:02.770801+0000","last_undegraded":"2024-03-18T15:40:02.770801+0000","last_fullsized":"2024-03-18T15:40:02.770801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770565+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:40:02.770565+0000","last_peered":"2024-03-18T15:40:02.770565+0000","last_clean":"2024-03-18T15:40:02.770565+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:40:02.770565+0000","last_undegraded":"2024-03-18T15:40:02.770565+0000","last_fullsized":"2024-03-18T15:40:02.770565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973930+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973930+0000","last_peered":"2024-03-18T15:40:02.973930+0000","last_clean":"2024-03-18T15:40:02.973930+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973930+0000","last_undegraded":"2024-03-18T15:40:02.973930+0000","last_fullsized":"2024-03-18T15:40:02.973930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638688+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:02.638688+0000","last_peered":"2024-03-18T15:40:02.638688+0000","last_clean":"2024-03-18T15:40:02.638688+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:02.638688+0000","last_undegraded":"2024-03-18T15:40:02.638688+0000","last_fullsized":"2024-03-18T15:40:02.638688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788937+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:01.788937+0000","last_peered":"2024-03-18T15:40:01.788937+0000","last_clean":"2024-03-18T15:40:01.788937+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:01.788937+0000","last_undegraded":"2024-03-18T15:40:01.788937+0000","last_fullsized":"2024-03-18T15:40:01.788937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770536+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:40:02.770536+0000","last_peered":"2024-03-18T15:40:02.770536+0000","last_clean":"2024-03-18T15:40:02.770536+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:40:02.770536+0000","last_undegraded":"2024-03-18T15:40:02.770536+0000","last_fullsized":"2024-03-18T15:40:02.770536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770431+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:40:02.770431+0000","last_peered":"2024-03-18T15:40:02.770431+0000","last_clean":"2024-03-18T15:40:02.770431+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:40:02.770431+0000","last_undegraded":"2024-03-18T15:40:02.770431+0000","last_fullsized":"2024-03-18T15:40:02.770431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637280+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:02.637280+0000","last_peered":"2024-03-18T15:40:02.637280+0000","last_clean":"2024-03-18T15:40:02.637280+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:02.637280+0000","last_undegraded":"2024-03-18T15:40:02.637280+0000","last_fullsized":"2024-03-18T15:40:02.637280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.275148+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:03.275148+0000","last_peered":"2024-03-18T15:40:03.275148+0000","last_clean":"2024-03-18T15:40:03.275148+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:03.275148+0000","last_undegraded":"2024-03-18T15:40:03.275148+0000","last_fullsized":"2024-03-18T15:40:03.275148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639140+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:02.639140+0000","last_peered":"2024-03-18T15:40:02.639140+0000","last_clean":"2024-03-18T15:40:02.639140+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:02.639140+0000","last_undegraded":"2024-03-18T15:40:02.639140+0000","last_fullsized":"2024-03-18T15:40:02.639140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770915+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:40:02.770915+0000","last_peered":"2024-03-18T15:40:02.770915+0000","last_clean":"2024-03-18T15:40:02.770915+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:40:02.770915+0000","last_undegraded":"2024-03-18T15:40:02.770915+0000","last_fullsized":"2024-03-18T15:40:02.770915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634707+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:02.634707+0000","last_peered":"2024-03-18T15:40:02.634707+0000","last_clean":"2024-03-18T15:40:02.634707+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:02.634707+0000","last_undegraded":"2024-03-18T15:40:02.634707+0000","last_fullsized":"2024-03-18T15:40:02.634707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788710+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:01.788710+0000","last_peered":"2024-03-18T15:40:01.788710+0000","last_clean":"2024-03-18T15:40:01.788710+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:01.788710+0000","last_undegraded":"2024-03-18T15:40:01.788710+0000","last_fullsized":"2024-03-18T15:40:01.788710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638755+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:02.638755+0000","last_peered":"2024-03-18T15:40:02.638755+0000","last_clean":"2024-03-18T15:40:02.638755+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:02.638755+0000","last_undegraded":"2024-03-18T15:40:02.638755+0000","last_fullsized":"2024-03-18T15:40:02.638755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":258,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.267786+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:05.267786+0000","last_peered":"2024-03-18T15:40:05.267786+0000","last_clean":"2024-03-18T15:40:05.267786+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:05.267786+0000","last_undegraded":"2024-03-18T15:40:05.267786+0000","last_fullsized":"2024-03-18T15:40:05.267786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":230,"num_read_kb":230,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770819+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:40:02.770819+0000","last_peered":"2024-03-18T15:40:02.770819+0000","last_clean":"2024-03-18T15:40:02.770819+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:40:02.770819+0000","last_undegraded":"2024-03-18T15:40:02.770819+0000","last_fullsized":"2024-03-18T15:40:02.770819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":300,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:05.268965+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:05.268965+0000","last_peered":"2024-03-18T15:40:05.268965+0000","last_clean":"2024-03-18T15:40:05.268965+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:05.268965+0000","last_undegraded":"2024-03-18T15:40:05.268965+0000","last_fullsized":"2024-03-18T15:40:05.268965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":189,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.649987+0000","last_change":"2024-03-18T15:40:02.649987+0000","last_active":"2024-03-18T15:40:02.649987+0000","last_peered":"2024-03-18T15:40:02.649987+0000","last_clean":"2024-03-18T15:40:02.649987+0000","last_became_active":"2024-03-18T15:40:02.640566+0000","last_became_peered":"2024-03-18T15:40:02.640566+0000","last_unstale":"2024-03-18T15:40:02.649987+0000","last_undegraded":"2024-03-18T15:40:02.649987+0000","last_fullsized":"2024-03-18T15:40:02.649987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:49:31.828795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788725+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:01.788725+0000","last_peered":"2024-03-18T15:40:01.788725+0000","last_clean":"2024-03-18T15:40:01.788725+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:01.788725+0000","last_undegraded":"2024-03-18T15:40:01.788725+0000","last_fullsized":"2024-03-18T15:40:01.788725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:03.270266+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:03.270266+0000","last_peered":"2024-03-18T15:40:03.270266+0000","last_clean":"2024-03-18T15:40:03.270266+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:03.270266+0000","last_undegraded":"2024-03-18T15:40:03.270266+0000","last_fullsized":"2024-03-18T15:40:03.270266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637233+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:02.637233+0000","last_peered":"2024-03-18T15:40:02.637233+0000","last_clean":"2024-03-18T15:40:02.637233+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:02.637233+0000","last_undegraded":"2024-03-18T15:40:02.637233+0000","last_fullsized":"2024-03-18T15:40:02.637233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637255+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:02.637255+0000","last_peered":"2024-03-18T15:40:02.637255+0000","last_clean":"2024-03-18T15:40:02.637255+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:02.637255+0000","last_undegraded":"2024-03-18T15:40:02.637255+0000","last_fullsized":"2024-03-18T15:40:02.637255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827860+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827860+0000","last_peered":"2024-03-18T15:40:02.827860+0000","last_clean":"2024-03-18T15:40:02.827860+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827860+0000","last_undegraded":"2024-03-18T15:40:02.827860+0000","last_fullsized":"2024-03-18T15:40:02.827860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639678+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639678+0000","last_peered":"2024-03-18T15:40:02.639678+0000","last_clean":"2024-03-18T15:40:02.639678+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639678+0000","last_undegraded":"2024-03-18T15:40:02.639678+0000","last_fullsized":"2024-03-18T15:40:02.639678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639330+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:02.639330+0000","last_peered":"2024-03-18T15:40:02.639330+0000","last_clean":"2024-03-18T15:40:02.639330+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:02.639330+0000","last_undegraded":"2024-03-18T15:40:02.639330+0000","last_fullsized":"2024-03-18T15:40:02.639330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639749+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639749+0000","last_peered":"2024-03-18T15:40:02.639749+0000","last_clean":"2024-03-18T15:40:02.639749+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639749+0000","last_undegraded":"2024-03-18T15:40:02.639749+0000","last_fullsized":"2024-03-18T15:40:02.639749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634497+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634497+0000","last_peered":"2024-03-18T15:40:02.634497+0000","last_clean":"2024-03-18T15:40:02.634497+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634497+0000","last_undegraded":"2024-03-18T15:40:02.634497+0000","last_fullsized":"2024-03-18T15:40:02.634497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":139,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788655+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:01.788655+0000","last_peered":"2024-03-18T15:40:01.788655+0000","last_clean":"2024-03-18T15:40:01.788655+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:01.788655+0000","last_undegraded":"2024-03-18T15:40:01.788655+0000","last_fullsized":"2024-03-18T15:40:01.788655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638875+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:02.638875+0000","last_peered":"2024-03-18T15:40:02.638875+0000","last_clean":"2024-03-18T15:40:02.638875+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:02.638875+0000","last_undegraded":"2024-03-18T15:40:02.638875+0000","last_fullsized":"2024-03-18T15:40:02.638875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973820+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973820+0000","last_peered":"2024-03-18T15:40:02.973820+0000","last_clean":"2024-03-18T15:40:02.973820+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973820+0000","last_undegraded":"2024-03-18T15:40:02.973820+0000","last_fullsized":"2024-03-18T15:40:02.973820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788613+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:01.788613+0000","last_peered":"2024-03-18T15:40:01.788613+0000","last_clean":"2024-03-18T15:40:01.788613+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:01.788613+0000","last_undegraded":"2024-03-18T15:40:01.788613+0000","last_fullsized":"2024-03-18T15:40:01.788613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829967+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.829967+0000","last_peered":"2024-03-18T15:40:02.829967+0000","last_clean":"2024-03-18T15:40:02.829967+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.829967+0000","last_undegraded":"2024-03-18T15:40:02.829967+0000","last_fullsized":"2024-03-18T15:40:02.829967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788812+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:01.788812+0000","last_peered":"2024-03-18T15:40:01.788812+0000","last_clean":"2024-03-18T15:40:01.788812+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:01.788812+0000","last_undegraded":"2024-03-18T15:40:01.788812+0000","last_fullsized":"2024-03-18T15:40:01.788812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639056+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:02.639056+0000","last_peered":"2024-03-18T15:40:02.639056+0000","last_clean":"2024-03-18T15:40:02.639056+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:02.639056+0000","last_undegraded":"2024-03-18T15:40:02.639056+0000","last_fullsized":"2024-03-18T15:40:02.639056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770696+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:40:02.770696+0000","last_peered":"2024-03-18T15:40:02.770696+0000","last_clean":"2024-03-18T15:40:02.770696+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:40:02.770696+0000","last_undegraded":"2024-03-18T15:40:02.770696+0000","last_fullsized":"2024-03-18T15:40:02.770696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638847+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:02.638847+0000","last_peered":"2024-03-18T15:40:02.638847+0000","last_clean":"2024-03-18T15:40:02.638847+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:02.638847+0000","last_undegraded":"2024-03-18T15:40:02.638847+0000","last_fullsized":"2024-03-18T15:40:02.638847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637056+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:02.637056+0000","last_peered":"2024-03-18T15:40:02.637056+0000","last_clean":"2024-03-18T15:40:02.637056+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:02.637056+0000","last_undegraded":"2024-03-18T15:40:02.637056+0000","last_fullsized":"2024-03-18T15:40:02.637056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639076+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639076+0000","last_peered":"2024-03-18T15:40:02.639076+0000","last_clean":"2024-03-18T15:40:02.639076+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639076+0000","last_undegraded":"2024-03-18T15:40:02.639076+0000","last_fullsized":"2024-03-18T15:40:02.639076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638975+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:02.638975+0000","last_peered":"2024-03-18T15:40:02.638975+0000","last_clean":"2024-03-18T15:40:02.638975+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:02.638975+0000","last_undegraded":"2024-03-18T15:40:02.638975+0000","last_fullsized":"2024-03-18T15:40:02.638975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639381+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:02.639381+0000","last_peered":"2024-03-18T15:40:02.639381+0000","last_clean":"2024-03-18T15:40:02.639381+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:02.639381+0000","last_undegraded":"2024-03-18T15:40:02.639381+0000","last_fullsized":"2024-03-18T15:40:02.639381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788849+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:01.788849+0000","last_peered":"2024-03-18T15:40:01.788849+0000","last_clean":"2024-03-18T15:40:01.788849+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:01.788849+0000","last_undegraded":"2024-03-18T15:40:01.788849+0000","last_fullsized":"2024-03-18T15:40:01.788849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637119+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:02.637119+0000","last_peered":"2024-03-18T15:40:02.637119+0000","last_clean":"2024-03-18T15:40:02.637119+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:02.637119+0000","last_undegraded":"2024-03-18T15:40:02.637119+0000","last_fullsized":"2024-03-18T15:40:02.637119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770759+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:40:02.770759+0000","last_peered":"2024-03-18T15:40:02.770759+0000","last_clean":"2024-03-18T15:40:02.770759+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:40:02.770759+0000","last_undegraded":"2024-03-18T15:40:02.770759+0000","last_fullsized":"2024-03-18T15:40:02.770759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788795+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:01.788795+0000","last_peered":"2024-03-18T15:40:01.788795+0000","last_clean":"2024-03-18T15:40:01.788795+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:01.788795+0000","last_undegraded":"2024-03-18T15:40:01.788795+0000","last_fullsized":"2024-03-18T15:40:01.788795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637235+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:02.637235+0000","last_peered":"2024-03-18T15:40:02.637235+0000","last_clean":"2024-03-18T15:40:02.637235+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:02.637235+0000","last_undegraded":"2024-03-18T15:40:02.637235+0000","last_fullsized":"2024-03-18T15:40:02.637235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639014+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:02.639014+0000","last_peered":"2024-03-18T15:40:02.639014+0000","last_clean":"2024-03-18T15:40:02.639014+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:02.639014+0000","last_undegraded":"2024-03-18T15:40:02.639014+0000","last_fullsized":"2024-03-18T15:40:02.639014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638711+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:02.638711+0000","last_peered":"2024-03-18T15:40:02.638711+0000","last_clean":"2024-03-18T15:40:02.638711+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:02.638711+0000","last_undegraded":"2024-03-18T15:40:02.638711+0000","last_fullsized":"2024-03-18T15:40:02.638711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-03-18T15:40:01.788850+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:01.788850+0000","last_peered":"2024-03-18T15:40:01.788850+0000","last_clean":"2024-03-18T15:40:01.788850+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:01.788850+0000","last_undegraded":"2024-03-18T15:40:01.788850+0000","last_fullsized":"2024-03-18T15:40:01.788850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637251+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:02.637251+0000","last_peered":"2024-03-18T15:40:02.637251+0000","last_clean":"2024-03-18T15:40:02.637251+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:02.637251+0000","last_undegraded":"2024-03-18T15:40:02.637251+0000","last_fullsized":"2024-03-18T15:40:02.637251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1177,"num_read_kb":675,"num_write":250,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":222349,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":478900,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":469581,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":479796,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":532009,"data_compressed":10649,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":477891,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920842,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":469750,"data_compressed":8781,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724613,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":1062047,"data_compressed":44378,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63507,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:08 smithi193 ceph-mon[36702]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-18T15:40:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:08 smithi193 ceph-mon[36702]: Cluster is now healthy 2024-03-18T15:40:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:08 smithi193 ceph-mon[36702]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:08 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/666908972' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-18T15:40:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:08 smithi193 ceph-mon[36702]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:08 smithi096 ceph-mon[31930]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-18T15:40:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:08 smithi096 ceph-mon[31930]: Cluster is now healthy 2024-03-18T15:40:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:08 smithi096 ceph-mon[31930]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:08 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/666908972' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-18T15:40:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:08 smithi096 ceph-mon[31930]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:08 smithi137 ceph-mon[35868]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-18T15:40:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:08 smithi137 ceph-mon[35868]: Cluster is now healthy 2024-03-18T15:40:09.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:08 smithi137 ceph-mon[35868]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:09.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:08 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/666908972' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-18T15:40:09.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:08 smithi137 ceph-mon[35868]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:09.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:09 smithi193 ceph-mon[36702]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:09.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:09 smithi096 ceph-mon[31930]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:10.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:09 smithi137 ceph-mon[35868]: pgmap v354: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:11.511 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- 172.21.15.96:0/2805073953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba81018f0 msgr2=0x7feba8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 --2- 172.21.15.96:0/2805073953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba81018f0 0x7feba8101d70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009900 tx=0x7feb9c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- 172.21.15.96:0/2805073953 shutdown_connections 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 --2- 172.21.15.96:0/2805073953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba8108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 --2- 172.21.15.96:0/2805073953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba81018f0 0x7feba8101d70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 --2- 172.21.15.96:0/2805073953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba8100700 0x7feba8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- 172.21.15.96:0/2805073953 >> 172.21.15.96:0/2805073953 conn(0x7feba80fbc70 msgr2=0x7feba80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- 172.21.15.96:0/2805073953 shutdown_connections 2024-03-18T15:40:11.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- 172.21.15.96:0/2805073953 wait complete. 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 Processor -- start 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.626+0000 7febadce8640 1 -- start start 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba81018f0 0x7feba819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba81a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feba810c3d0 con 0x7feba8100700 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feba810c250 con 0x7feba81018f0 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feba810c550 con 0x7feba81022b0 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40394/0 (socket says 172.21.15.96:40394) 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 -- 172.21.15.96:0/3229447243 learned_addr learned my addr 172.21.15.96:0/3229447243 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba6ffd640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba81018f0 0x7feba819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba7fff640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba81a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 -- 172.21.15.96:0/3229447243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 msgr2=0x7feba81a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba81a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 -- 172.21.15.96:0/3229447243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba81018f0 msgr2=0x7feba819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba81018f0 0x7feba819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 -- 172.21.15.96:0/3229447243 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba81a1910 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba7fff640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba81a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba77fe640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9c0 tx=0x7feb9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800cd80 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba81a1c00 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb980035f0 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98010690 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.627+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81a2150 con 0x7feba8100700 2024-03-18T15:40:11.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.628+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba81a2320 con 0x7feba8100700 2024-03-18T15:40:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.629+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7feb980037a0 con 0x7feba8100700 2024-03-18T15:40:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.630+0000 7feba4ff9640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 0x7feb900a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.630+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7feb980cf0f0 con 0x7feba8100700 2024-03-18T15:40:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.630+0000 7feba6ffd640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 0x7feb900a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.630+0000 7feba6ffd640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 0x7feb900a4d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02c240 tx=0x7feb9c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:11.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.634+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7feb9809c000 con 0x7feba8100700 2024-03-18T15:40:11.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.805+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feba81a24f0 con 0x7feb900a2890 2024-03-18T15:40:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.813+0000 7feba4ff9640 1 -- 172.21.15.96:0/3229447243 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259188 (secure 0 0 0) 0x7feba81a24f0 con 0x7feb900a2890 2024-03-18T15:40:11.817 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:11.818 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:11.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 msgr2=0x7feb900a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 0x7feb900a4d50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feb9c02c240 tx=0x7feb9c02df90 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 msgr2=0x7feba8199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb9800e9c0 tx=0x7feb9800ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 shutdown_connections 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feb900a2890 0x7feb900a4d50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feba81022b0 0x7feba81a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feba81018f0 0x7feba819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 --2- 172.21.15.96:0/3229447243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feba8100700 0x7feba8199d00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 >> 172.21.15.96:0/3229447243 conn(0x7feba80fbc70 msgr2=0x7feba8068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 shutdown_connections 2024-03-18T15:40:11.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.816+0000 7febadce8640 1 -- 172.21.15.96:0/3229447243 wait complete. 2024-03-18T15:40:11.828 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-03-18T15:40:11.475369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667610,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1354,"num_read_kb":810,"num_write":437,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":858,"ondisk_log_size":858,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365524,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576300,"statfs":{"total":1151940427776,"available":1151566131200,"internally_reserved":0,"allocated":12447744,"data_stored":7729554,"data_compressed":252927,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9291,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":62,"num_write":14,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.843118"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637350+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:02.637350+0000","last_peered":"2024-03-18T15:40:02.637350+0000","last_clean":"2024-03-18T15:40:02.637350+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:02.637350+0000","last_undegraded":"2024-03-18T15:40:02.637350+0000","last_fullsized":"2024-03-18T15:40:02.637350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770801+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:40:02.770801+0000","last_peered":"2024-03-18T15:40:02.770801+0000","last_clean":"2024-03-18T15:40:02.770801+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:40:02.770801+0000","last_undegraded":"2024-03-18T15:40:02.770801+0000","last_fullsized":"2024-03-18T15:40:02.770801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770565+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:40:02.770565+0000","last_peered":"2024-03-18T15:40:02.770565+0000","last_clean":"2024-03-18T15:40:02.770565+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:40:02.770565+0000","last_undegraded":"2024-03-18T15:40:02.770565+0000","last_fullsized":"2024-03-18T15:40:02.770565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973977+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973977+0000","last_peered":"2024-03-18T15:40:02.973977+0000","last_clean":"2024-03-18T15:40:02.973977+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973977+0000","last_undegraded":"2024-03-18T15:40:02.973977+0000","last_fullsized":"2024-03-18T15:40:02.973977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638688+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:02.638688+0000","last_peered":"2024-03-18T15:40:02.638688+0000","last_clean":"2024-03-18T15:40:02.638688+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:02.638688+0000","last_undegraded":"2024-03-18T15:40:02.638688+0000","last_fullsized":"2024-03-18T15:40:02.638688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634361+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:02.634361+0000","last_peered":"2024-03-18T15:40:02.634361+0000","last_clean":"2024-03-18T15:40:02.634361+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:02.634361+0000","last_undegraded":"2024-03-18T15:40:02.634361+0000","last_fullsized":"2024-03-18T15:40:02.634361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770536+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:40:02.770536+0000","last_peered":"2024-03-18T15:40:02.770536+0000","last_clean":"2024-03-18T15:40:02.770536+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:40:02.770536+0000","last_undegraded":"2024-03-18T15:40:02.770536+0000","last_fullsized":"2024-03-18T15:40:02.770536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770431+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:40:02.770431+0000","last_peered":"2024-03-18T15:40:02.770431+0000","last_clean":"2024-03-18T15:40:02.770431+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:40:02.770431+0000","last_undegraded":"2024-03-18T15:40:02.770431+0000","last_fullsized":"2024-03-18T15:40:02.770431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637280+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:02.637280+0000","last_peered":"2024-03-18T15:40:02.637280+0000","last_clean":"2024-03-18T15:40:02.637280+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:02.637280+0000","last_undegraded":"2024-03-18T15:40:02.637280+0000","last_fullsized":"2024-03-18T15:40:02.637280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:09.281660+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:09.281660+0000","last_peered":"2024-03-18T15:40:09.281660+0000","last_clean":"2024-03-18T15:40:09.281660+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:09.281660+0000","last_undegraded":"2024-03-18T15:40:09.281660+0000","last_fullsized":"2024-03-18T15:40:09.281660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639140+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:02.639140+0000","last_peered":"2024-03-18T15:40:02.639140+0000","last_clean":"2024-03-18T15:40:02.639140+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:02.639140+0000","last_undegraded":"2024-03-18T15:40:02.639140+0000","last_fullsized":"2024-03-18T15:40:02.639140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770915+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:40:02.770915+0000","last_peered":"2024-03-18T15:40:02.770915+0000","last_clean":"2024-03-18T15:40:02.770915+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:40:02.770915+0000","last_undegraded":"2024-03-18T15:40:02.770915+0000","last_fullsized":"2024-03-18T15:40:02.770915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:06.269368+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:06.269368+0000","last_peered":"2024-03-18T15:40:06.269368+0000","last_clean":"2024-03-18T15:40:06.269368+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:06.269368+0000","last_undegraded":"2024-03-18T15:40:06.269368+0000","last_fullsized":"2024-03-18T15:40:06.269368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634169+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:02.634169+0000","last_peered":"2024-03-18T15:40:02.634169+0000","last_clean":"2024-03-18T15:40:02.634169+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:02.634169+0000","last_undegraded":"2024-03-18T15:40:02.634169+0000","last_fullsized":"2024-03-18T15:40:02.634169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638755+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:02.638755+0000","last_peered":"2024-03-18T15:40:02.638755+0000","last_clean":"2024-03-18T15:40:02.638755+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:02.638755+0000","last_undegraded":"2024-03-18T15:40:02.638755+0000","last_fullsized":"2024-03-18T15:40:02.638755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":268,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:10.272934+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:10.272934+0000","last_peered":"2024-03-18T15:40:10.272934+0000","last_clean":"2024-03-18T15:40:10.272934+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:10.272934+0000","last_undegraded":"2024-03-18T15:40:10.272934+0000","last_fullsized":"2024-03-18T15:40:10.272934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770819+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:40:02.770819+0000","last_peered":"2024-03-18T15:40:02.770819+0000","last_clean":"2024-03-18T15:40:02.770819+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:40:02.770819+0000","last_undegraded":"2024-03-18T15:40:02.770819+0000","last_fullsized":"2024-03-18T15:40:02.770819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":310,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:10.273239+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:10.273239+0000","last_peered":"2024-03-18T15:40:10.273239+0000","last_clean":"2024-03-18T15:40:10.273239+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:10.273239+0000","last_undegraded":"2024-03-18T15:40:10.273239+0000","last_fullsized":"2024-03-18T15:40:10.273239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.649987+0000","last_change":"2024-03-18T15:40:02.649987+0000","last_active":"2024-03-18T15:40:02.649987+0000","last_peered":"2024-03-18T15:40:02.649987+0000","last_clean":"2024-03-18T15:40:02.649987+0000","last_became_active":"2024-03-18T15:40:02.640566+0000","last_became_peered":"2024-03-18T15:40:02.640566+0000","last_unstale":"2024-03-18T15:40:02.649987+0000","last_undegraded":"2024-03-18T15:40:02.649987+0000","last_fullsized":"2024-03-18T15:40:02.649987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:49:31.828795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634164+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:02.634164+0000","last_peered":"2024-03-18T15:40:02.634164+0000","last_clean":"2024-03-18T15:40:02.634164+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:02.634164+0000","last_undegraded":"2024-03-18T15:40:02.634164+0000","last_fullsized":"2024-03-18T15:40:02.634164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:07.275296+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:07.275296+0000","last_peered":"2024-03-18T15:40:07.275296+0000","last_clean":"2024-03-18T15:40:07.275296+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:07.275296+0000","last_undegraded":"2024-03-18T15:40:07.275296+0000","last_fullsized":"2024-03-18T15:40:07.275296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637233+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:02.637233+0000","last_peered":"2024-03-18T15:40:02.637233+0000","last_clean":"2024-03-18T15:40:02.637233+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:02.637233+0000","last_undegraded":"2024-03-18T15:40:02.637233+0000","last_fullsized":"2024-03-18T15:40:02.637233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637255+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:02.637255+0000","last_peered":"2024-03-18T15:40:02.637255+0000","last_clean":"2024-03-18T15:40:02.637255+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:02.637255+0000","last_undegraded":"2024-03-18T15:40:02.637255+0000","last_fullsized":"2024-03-18T15:40:02.637255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827933+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827933+0000","last_peered":"2024-03-18T15:40:02.827933+0000","last_clean":"2024-03-18T15:40:02.827933+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827933+0000","last_undegraded":"2024-03-18T15:40:02.827933+0000","last_fullsized":"2024-03-18T15:40:02.827933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639726+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639726+0000","last_peered":"2024-03-18T15:40:02.639726+0000","last_clean":"2024-03-18T15:40:02.639726+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639726+0000","last_undegraded":"2024-03-18T15:40:02.639726+0000","last_fullsized":"2024-03-18T15:40:02.639726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:08.281001+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:08.281001+0000","last_peered":"2024-03-18T15:40:08.281001+0000","last_clean":"2024-03-18T15:40:08.281001+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:08.281001+0000","last_undegraded":"2024-03-18T15:40:08.281001+0000","last_fullsized":"2024-03-18T15:40:08.281001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639799+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639799+0000","last_peered":"2024-03-18T15:40:02.639799+0000","last_clean":"2024-03-18T15:40:02.639799+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639799+0000","last_undegraded":"2024-03-18T15:40:02.639799+0000","last_fullsized":"2024-03-18T15:40:02.639799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634550+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634550+0000","last_peered":"2024-03-18T15:40:02.634550+0000","last_clean":"2024-03-18T15:40:02.634550+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634550+0000","last_undegraded":"2024-03-18T15:40:02.634550+0000","last_fullsized":"2024-03-18T15:40:02.634550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:06.273966+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:06.273966+0000","last_peered":"2024-03-18T15:40:06.273966+0000","last_clean":"2024-03-18T15:40:06.273966+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:06.273966+0000","last_undegraded":"2024-03-18T15:40:06.273966+0000","last_fullsized":"2024-03-18T15:40:06.273966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638875+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:02.638875+0000","last_peered":"2024-03-18T15:40:02.638875+0000","last_clean":"2024-03-18T15:40:02.638875+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:02.638875+0000","last_undegraded":"2024-03-18T15:40:02.638875+0000","last_fullsized":"2024-03-18T15:40:02.638875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973870+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973870+0000","last_peered":"2024-03-18T15:40:02.973870+0000","last_clean":"2024-03-18T15:40:02.973870+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973870+0000","last_undegraded":"2024-03-18T15:40:02.973870+0000","last_fullsized":"2024-03-18T15:40:02.973870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634027+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:02.634027+0000","last_peered":"2024-03-18T15:40:02.634027+0000","last_clean":"2024-03-18T15:40:02.634027+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:02.634027+0000","last_undegraded":"2024-03-18T15:40:02.634027+0000","last_fullsized":"2024-03-18T15:40:02.634027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830021+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.830021+0000","last_peered":"2024-03-18T15:40:02.830021+0000","last_clean":"2024-03-18T15:40:02.830021+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.830021+0000","last_undegraded":"2024-03-18T15:40:02.830021+0000","last_fullsized":"2024-03-18T15:40:02.830021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634273+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:02.634273+0000","last_peered":"2024-03-18T15:40:02.634273+0000","last_clean":"2024-03-18T15:40:02.634273+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:02.634273+0000","last_undegraded":"2024-03-18T15:40:02.634273+0000","last_fullsized":"2024-03-18T15:40:02.634273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639056+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:02.639056+0000","last_peered":"2024-03-18T15:40:02.639056+0000","last_clean":"2024-03-18T15:40:02.639056+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:02.639056+0000","last_undegraded":"2024-03-18T15:40:02.639056+0000","last_fullsized":"2024-03-18T15:40:02.639056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770696+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:40:02.770696+0000","last_peered":"2024-03-18T15:40:02.770696+0000","last_clean":"2024-03-18T15:40:02.770696+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:40:02.770696+0000","last_undegraded":"2024-03-18T15:40:02.770696+0000","last_fullsized":"2024-03-18T15:40:02.770696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638847+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:02.638847+0000","last_peered":"2024-03-18T15:40:02.638847+0000","last_clean":"2024-03-18T15:40:02.638847+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:02.638847+0000","last_undegraded":"2024-03-18T15:40:02.638847+0000","last_fullsized":"2024-03-18T15:40:02.638847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637056+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:02.637056+0000","last_peered":"2024-03-18T15:40:02.637056+0000","last_clean":"2024-03-18T15:40:02.637056+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:02.637056+0000","last_undegraded":"2024-03-18T15:40:02.637056+0000","last_fullsized":"2024-03-18T15:40:02.637056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639128+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639128+0000","last_peered":"2024-03-18T15:40:02.639128+0000","last_clean":"2024-03-18T15:40:02.639128+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639128+0000","last_undegraded":"2024-03-18T15:40:02.639128+0000","last_fullsized":"2024-03-18T15:40:02.639128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638975+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:02.638975+0000","last_peered":"2024-03-18T15:40:02.638975+0000","last_clean":"2024-03-18T15:40:02.638975+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:02.638975+0000","last_undegraded":"2024-03-18T15:40:02.638975+0000","last_fullsized":"2024-03-18T15:40:02.638975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:07.289529+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:07.289529+0000","last_peered":"2024-03-18T15:40:07.289529+0000","last_clean":"2024-03-18T15:40:07.289529+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:07.289529+0000","last_undegraded":"2024-03-18T15:40:07.289529+0000","last_fullsized":"2024-03-18T15:40:07.289529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634335+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:02.634335+0000","last_peered":"2024-03-18T15:40:02.634335+0000","last_clean":"2024-03-18T15:40:02.634335+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:02.634335+0000","last_undegraded":"2024-03-18T15:40:02.634335+0000","last_fullsized":"2024-03-18T15:40:02.634335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637119+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:02.637119+0000","last_peered":"2024-03-18T15:40:02.637119+0000","last_clean":"2024-03-18T15:40:02.637119+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:02.637119+0000","last_undegraded":"2024-03-18T15:40:02.637119+0000","last_fullsized":"2024-03-18T15:40:02.637119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770759+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:40:02.770759+0000","last_peered":"2024-03-18T15:40:02.770759+0000","last_clean":"2024-03-18T15:40:02.770759+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:40:02.770759+0000","last_undegraded":"2024-03-18T15:40:02.770759+0000","last_fullsized":"2024-03-18T15:40:02.770759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634266+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:02.634266+0000","last_peered":"2024-03-18T15:40:02.634266+0000","last_clean":"2024-03-18T15:40:02.634266+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:02.634266+0000","last_undegraded":"2024-03-18T15:40:02.634266+0000","last_fullsized":"2024-03-18T15:40:02.634266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637235+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:02.637235+0000","last_peered":"2024-03-18T15:40:02.637235+0000","last_clean":"2024-03-18T15:40:02.637235+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:02.637235+0000","last_undegraded":"2024-03-18T15:40:02.637235+0000","last_fullsized":"2024-03-18T15:40:02.637235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639014+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:02.639014+0000","last_peered":"2024-03-18T15:40:02.639014+0000","last_clean":"2024-03-18T15:40:02.639014+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:02.639014+0000","last_undegraded":"2024-03-18T15:40:02.639014+0000","last_fullsized":"2024-03-18T15:40:02.639014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638711+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:02.638711+0000","last_peered":"2024-03-18T15:40:02.638711+0000","last_clean":"2024-03-18T15:40:02.638711+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:02.638711+0000","last_undegraded":"2024-03-18T15:40:02.638711+0000","last_fullsized":"2024-03-18T15:40:02.638711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634382+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:02.634382+0000","last_peered":"2024-03-18T15:40:02.634382+0000","last_clean":"2024-03-18T15:40:02.634382+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:02.634382+0000","last_undegraded":"2024-03-18T15:40:02.634382+0000","last_fullsized":"2024-03-18T15:40:02.634382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637251+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:02.637251+0000","last_peered":"2024-03-18T15:40:02.637251+0000","last_clean":"2024-03-18T15:40:02.637251+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:02.637251+0000","last_undegraded":"2024-03-18T15:40:02.637251+0000","last_fullsized":"2024-03-18T15:40:02.637251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77008,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1228,"num_read_kb":701,"num_write":254,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":231024,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":478900,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":478066,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":479796,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":549409,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":477891,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":478235,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1070532,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72182,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:11.832 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-18T15:40:11.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:40:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 -- 172.21.15.96:0/1026285144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b01074e0 msgr2=0x7f65b0109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 --2- 172.21.15.96:0/1026285144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b01074e0 0x7f65b0109930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009900 tx=0x7f65a802bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 -- 172.21.15.96:0/1026285144 shutdown_connections 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 --2- 172.21.15.96:0/1026285144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b01074e0 0x7f65b0109930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 --2- 172.21.15.96:0/1026285144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0104bb0 0x7f65b0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 --2- 172.21.15.96:0/1026285144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b0102280 0x7f65b0104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 -- 172.21.15.96:0/1026285144 >> 172.21.15.96:0/1026285144 conn(0x7f65b00fbc70 msgr2=0x7f65b00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 -- 172.21.15.96:0/1026285144 shutdown_connections 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.968+0000 7f65b78c0640 1 -- 172.21.15.96:0/1026285144 wait complete. 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 Processor -- start 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 -- start start 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 0x7f65b0197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b01074e0 0x7f65b019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010d4a0 con 0x7f65b0104bb0 2024-03-18T15:40:11.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010d320 con 0x7f65b0102280 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b78c0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f65b010d620 con 0x7f65b01074e0 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b4e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b565c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 0x7f65b0197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b4e5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40412/0 (socket says 172.21.15.96:40412) 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b4e5b640 1 -- 172.21.15.96:0/2877720120 learned_addr learned my addr 172.21.15.96:0/2877720120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.969+0000 7f65b5e5d640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b01074e0 0x7f65b019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 -- 172.21.15.96:0/2877720120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b01074e0 msgr2=0x7f65b019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b01074e0 0x7f65b019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 -- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 msgr2=0x7f65b0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 -- 172.21.15.96:0/2877720120 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b019f710 con 0x7f65b0102280 2024-03-18T15:40:11.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b4e5b640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b565c640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 0x7f65b0197af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009fd0 tx=0x7f65a000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0004b20 con 0x7f65b0102280 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a0004cc0 con 0x7f65b0102280 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b019fa00 con 0x7f65b0102280 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a00107b0 con 0x7f65b0102280 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.970+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b019ff00 con 0x7f65b0102280 2024-03-18T15:40:11.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.971+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b00fd760 con 0x7f65b0102280 2024-03-18T15:40:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.972+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f65a0005020 con 0x7f65b0102280 2024-03-18T15:40:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.972+0000 7f65ae7fc640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 0x7f65980a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.973+0000 7f65b4e5b640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 0x7f65980a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.973+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f65a00cf6e0 con 0x7f65b0102280 2024-03-18T15:40:11.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.973+0000 7f65b4e5b640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 0x7f65980a4d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65b0199000 tx=0x7f65a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:11.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:11.976+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f65a009c5d0 con 0x7f65b0102280 2024-03-18T15:40:12.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.148+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 --> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65b0061c20 con 0x7f65980a2890 2024-03-18T15:40:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.155+0000 7f65ae7fc640 1 -- 172.21.15.96:0/2877720120 <== mgr.14152 v2:172.21.15.96:6800/3036369138 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259188 (secure 0 0 0) 0x7f65b0061c20 con 0x7f65980a2890 2024-03-18T15:40:12.159 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:12.160 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 msgr2=0x7f65980a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 0x7f65980a4d50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65b0199000 tx=0x7f65a4005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 msgr2=0x7f65b0197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 0x7f65b0197af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65a0009fd0 tx=0x7f65a000ef90 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 shutdown_connections 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f65980a2890 0x7f65980a4d50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65b01074e0 0x7f65b019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65b0104bb0 0x7f65b0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 --2- 172.21.15.96:0/2877720120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65b0102280 0x7f65b0197af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 >> 172.21.15.96:0/2877720120 conn(0x7f65b00fbc70 msgr2=0x7f65b0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 shutdown_connections 2024-03-18T15:40:12.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.158+0000 7f65b78c0640 1 -- 172.21.15.96:0/2877720120 wait complete. 2024-03-18T15:40:12.170 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-03-18T15:40:11.475369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":667610,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1354,"num_read_kb":810,"num_write":437,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180826,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":858,"ondisk_log_size":858,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365524,"kb_used_data":12156,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576300,"statfs":{"total":1151940427776,"available":1151566131200,"internally_reserved":0,"allocated":12447744,"data_stored":7729554,"data_compressed":252927,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9291,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":62,"num_write":14,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.843118"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637350+0000","last_change":"2024-03-18T15:38:04.697603+0000","last_active":"2024-03-18T15:40:02.637350+0000","last_peered":"2024-03-18T15:40:02.637350+0000","last_clean":"2024-03-18T15:40:02.637350+0000","last_became_active":"2024-03-18T15:38:04.697231+0000","last_became_peered":"2024-03-18T15:38:04.697231+0000","last_unstale":"2024-03-18T15:40:02.637350+0000","last_undegraded":"2024-03-18T15:40:02.637350+0000","last_fullsized":"2024-03-18T15:40:02.637350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:20:03.766843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770801+0000","last_change":"2024-03-18T15:38:04.691203+0000","last_active":"2024-03-18T15:40:02.770801+0000","last_peered":"2024-03-18T15:40:02.770801+0000","last_clean":"2024-03-18T15:40:02.770801+0000","last_became_active":"2024-03-18T15:38:04.690909+0000","last_became_peered":"2024-03-18T15:38:04.690909+0000","last_unstale":"2024-03-18T15:40:02.770801+0000","last_undegraded":"2024-03-18T15:40:02.770801+0000","last_fullsized":"2024-03-18T15:40:02.770801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:19:47.018144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770565+0000","last_change":"2024-03-18T15:38:04.688109+0000","last_active":"2024-03-18T15:40:02.770565+0000","last_peered":"2024-03-18T15:40:02.770565+0000","last_clean":"2024-03-18T15:40:02.770565+0000","last_became_active":"2024-03-18T15:38:04.687785+0000","last_became_peered":"2024-03-18T15:38:04.687785+0000","last_unstale":"2024-03-18T15:40:02.770565+0000","last_undegraded":"2024-03-18T15:40:02.770565+0000","last_fullsized":"2024-03-18T15:40:02.770565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:48:36.779409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973977+0000","last_change":"2024-03-18T15:38:04.697218+0000","last_active":"2024-03-18T15:40:02.973977+0000","last_peered":"2024-03-18T15:40:02.973977+0000","last_clean":"2024-03-18T15:40:02.973977+0000","last_became_active":"2024-03-18T15:38:04.696800+0000","last_became_peered":"2024-03-18T15:38:04.696800+0000","last_unstale":"2024-03-18T15:40:02.973977+0000","last_undegraded":"2024-03-18T15:40:02.973977+0000","last_fullsized":"2024-03-18T15:40:02.973977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:44:41.384090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829655+0000","last_change":"2024-03-18T15:38:04.687378+0000","last_active":"2024-03-18T15:40:02.829655+0000","last_peered":"2024-03-18T15:40:02.829655+0000","last_clean":"2024-03-18T15:40:02.829655+0000","last_became_active":"2024-03-18T15:38:04.686690+0000","last_became_peered":"2024-03-18T15:38:04.686690+0000","last_unstale":"2024-03-18T15:40:02.829655+0000","last_undegraded":"2024-03-18T15:40:02.829655+0000","last_fullsized":"2024-03-18T15:40:02.829655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:13:32.878403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639647+0000","last_change":"2024-03-18T15:38:04.690744+0000","last_active":"2024-03-18T15:40:02.639647+0000","last_peered":"2024-03-18T15:40:02.639647+0000","last_clean":"2024-03-18T15:40:02.639647+0000","last_became_active":"2024-03-18T15:38:04.690356+0000","last_became_peered":"2024-03-18T15:38:04.690356+0000","last_unstale":"2024-03-18T15:40:02.639647+0000","last_undegraded":"2024-03-18T15:40:02.639647+0000","last_fullsized":"2024-03-18T15:40:02.639647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:59.210443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638856+0000","last_change":"2024-03-18T15:38:04.691452+0000","last_active":"2024-03-18T15:40:02.638856+0000","last_peered":"2024-03-18T15:40:02.638856+0000","last_clean":"2024-03-18T15:40:02.638856+0000","last_became_active":"2024-03-18T15:38:04.690944+0000","last_became_peered":"2024-03-18T15:38:04.690944+0000","last_unstale":"2024-03-18T15:40:02.638856+0000","last_undegraded":"2024-03-18T15:40:02.638856+0000","last_fullsized":"2024-03-18T15:40:02.638856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:02:34.479514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638688+0000","last_change":"2024-03-18T15:38:04.694583+0000","last_active":"2024-03-18T15:40:02.638688+0000","last_peered":"2024-03-18T15:40:02.638688+0000","last_clean":"2024-03-18T15:40:02.638688+0000","last_became_active":"2024-03-18T15:38:04.693524+0000","last_became_peered":"2024-03-18T15:38:04.693524+0000","last_unstale":"2024-03-18T15:40:02.638688+0000","last_undegraded":"2024-03-18T15:40:02.638688+0000","last_fullsized":"2024-03-18T15:40:02.638688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:26.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974241+0000","last_change":"2024-03-18T15:38:04.703756+0000","last_active":"2024-03-18T15:40:02.974241+0000","last_peered":"2024-03-18T15:40:02.974241+0000","last_clean":"2024-03-18T15:40:02.974241+0000","last_became_active":"2024-03-18T15:38:04.703432+0000","last_became_peered":"2024-03-18T15:38:04.703432+0000","last_unstale":"2024-03-18T15:40:02.974241+0000","last_undegraded":"2024-03-18T15:40:02.974241+0000","last_fullsized":"2024-03-18T15:40:02.974241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:09.152618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639954+0000","last_change":"2024-03-18T15:38:04.689672+0000","last_active":"2024-03-18T15:40:02.639954+0000","last_peered":"2024-03-18T15:40:02.639954+0000","last_clean":"2024-03-18T15:40:02.639954+0000","last_became_active":"2024-03-18T15:38:04.689233+0000","last_became_peered":"2024-03-18T15:38:04.689233+0000","last_unstale":"2024-03-18T15:40:02.639954+0000","last_undegraded":"2024-03-18T15:40:02.639954+0000","last_fullsized":"2024-03-18T15:40:02.639954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:50:56.704473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638969+0000","last_change":"2024-03-18T15:38:04.696664+0000","last_active":"2024-03-18T15:40:02.638969+0000","last_peered":"2024-03-18T15:40:02.638969+0000","last_clean":"2024-03-18T15:40:02.638969+0000","last_became_active":"2024-03-18T15:38:04.696286+0000","last_became_peered":"2024-03-18T15:38:04.696286+0000","last_unstale":"2024-03-18T15:40:02.638969+0000","last_undegraded":"2024-03-18T15:40:02.638969+0000","last_fullsized":"2024-03-18T15:40:02.638969+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:29:12.538826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973810+0000","last_change":"2024-03-18T15:38:04.690304+0000","last_active":"2024-03-18T15:40:02.973810+0000","last_peered":"2024-03-18T15:40:02.973810+0000","last_clean":"2024-03-18T15:40:02.973810+0000","last_became_active":"2024-03-18T15:38:04.689957+0000","last_became_peered":"2024-03-18T15:38:04.689957+0000","last_unstale":"2024-03-18T15:40:02.973810+0000","last_undegraded":"2024-03-18T15:40:02.973810+0000","last_fullsized":"2024-03-18T15:40:02.973810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:05:30.954880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634361+0000","last_change":"2024-03-18T15:38:04.687036+0000","last_active":"2024-03-18T15:40:02.634361+0000","last_peered":"2024-03-18T15:40:02.634361+0000","last_clean":"2024-03-18T15:40:02.634361+0000","last_became_active":"2024-03-18T15:38:04.686727+0000","last_became_peered":"2024-03-18T15:38:04.686727+0000","last_unstale":"2024-03-18T15:40:02.634361+0000","last_undegraded":"2024-03-18T15:40:02.634361+0000","last_fullsized":"2024-03-18T15:40:02.634361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:18.306861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770536+0000","last_change":"2024-03-18T15:38:04.690817+0000","last_active":"2024-03-18T15:40:02.770536+0000","last_peered":"2024-03-18T15:40:02.770536+0000","last_clean":"2024-03-18T15:40:02.770536+0000","last_became_active":"2024-03-18T15:38:04.690540+0000","last_became_peered":"2024-03-18T15:38:04.690540+0000","last_unstale":"2024-03-18T15:40:02.770536+0000","last_undegraded":"2024-03-18T15:40:02.770536+0000","last_fullsized":"2024-03-18T15:40:02.770536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:24:51.696459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770431+0000","last_change":"2024-03-18T15:38:04.693071+0000","last_active":"2024-03-18T15:40:02.770431+0000","last_peered":"2024-03-18T15:40:02.770431+0000","last_clean":"2024-03-18T15:40:02.770431+0000","last_became_active":"2024-03-18T15:38:04.692758+0000","last_became_peered":"2024-03-18T15:38:04.692758+0000","last_unstale":"2024-03-18T15:40:02.770431+0000","last_undegraded":"2024-03-18T15:40:02.770431+0000","last_fullsized":"2024-03-18T15:40:02.770431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:25:21.418134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973369+0000","last_change":"2024-03-18T15:38:04.695045+0000","last_active":"2024-03-18T15:40:02.973369+0000","last_peered":"2024-03-18T15:40:02.973369+0000","last_clean":"2024-03-18T15:40:02.973369+0000","last_became_active":"2024-03-18T15:38:04.694674+0000","last_became_peered":"2024-03-18T15:38:04.694674+0000","last_unstale":"2024-03-18T15:40:02.973369+0000","last_undegraded":"2024-03-18T15:40:02.973369+0000","last_fullsized":"2024-03-18T15:40:02.973369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:31:35.520093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637280+0000","last_change":"2024-03-18T15:38:04.696640+0000","last_active":"2024-03-18T15:40:02.637280+0000","last_peered":"2024-03-18T15:40:02.637280+0000","last_clean":"2024-03-18T15:40:02.637280+0000","last_became_active":"2024-03-18T15:38:04.696052+0000","last_became_peered":"2024-03-18T15:38:04.696052+0000","last_unstale":"2024-03-18T15:40:02.637280+0000","last_undegraded":"2024-03-18T15:40:02.637280+0000","last_fullsized":"2024-03-18T15:40:02.637280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:52:41.604653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827961+0000","last_change":"2024-03-18T15:38:04.689138+0000","last_active":"2024-03-18T15:40:02.827961+0000","last_peered":"2024-03-18T15:40:02.827961+0000","last_clean":"2024-03-18T15:40:02.827961+0000","last_became_active":"2024-03-18T15:38:04.688808+0000","last_became_peered":"2024-03-18T15:38:04.688808+0000","last_unstale":"2024-03-18T15:40:02.827961+0000","last_undegraded":"2024-03-18T15:40:02.827961+0000","last_fullsized":"2024-03-18T15:40:02.827961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:51:33.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639323+0000","last_change":"2024-03-18T15:38:04.684736+0000","last_active":"2024-03-18T15:40:02.639323+0000","last_peered":"2024-03-18T15:40:02.639323+0000","last_clean":"2024-03-18T15:40:02.639323+0000","last_became_active":"2024-03-18T15:38:04.684385+0000","last_became_peered":"2024-03-18T15:38:04.684385+0000","last_unstale":"2024-03-18T15:40:02.639323+0000","last_undegraded":"2024-03-18T15:40:02.639323+0000","last_fullsized":"2024-03-18T15:40:02.639323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:22:13.679581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:09.281660+0000","last_change":"2024-03-18T15:38:03.827467+0000","last_active":"2024-03-18T15:40:09.281660+0000","last_peered":"2024-03-18T15:40:09.281660+0000","last_clean":"2024-03-18T15:40:09.281660+0000","last_became_active":"2024-03-18T15:38:03.827147+0000","last_became_peered":"2024-03-18T15:38:03.827147+0000","last_unstale":"2024-03-18T15:40:09.281660+0000","last_undegraded":"2024-03-18T15:40:09.281660+0000","last_fullsized":"2024-03-18T15:40:09.281660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:12:33.179097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827862+0000","last_change":"2024-03-18T15:39:02.571912+0000","last_active":"2024-03-18T15:40:02.827862+0000","last_peered":"2024-03-18T15:40:02.827862+0000","last_clean":"2024-03-18T15:40:02.827862+0000","last_became_active":"2024-03-18T15:39:02.571557+0000","last_became_peered":"2024-03-18T15:39:02.571557+0000","last_unstale":"2024-03-18T15:40:02.827862+0000","last_undegraded":"2024-03-18T15:40:02.827862+0000","last_fullsized":"2024-03-18T15:40:02.827862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:27:03.528899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639140+0000","last_change":"2024-03-18T15:38:03.832884+0000","last_active":"2024-03-18T15:40:02.639140+0000","last_peered":"2024-03-18T15:40:02.639140+0000","last_clean":"2024-03-18T15:40:02.639140+0000","last_became_active":"2024-03-18T15:38:03.832456+0000","last_became_peered":"2024-03-18T15:38:03.832456+0000","last_unstale":"2024-03-18T15:40:02.639140+0000","last_undegraded":"2024-03-18T15:40:02.639140+0000","last_fullsized":"2024-03-18T15:40:02.639140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:11:06.593962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770915+0000","last_change":"2024-03-18T15:38:04.702801+0000","last_active":"2024-03-18T15:40:02.770915+0000","last_peered":"2024-03-18T15:40:02.770915+0000","last_clean":"2024-03-18T15:40:02.770915+0000","last_became_active":"2024-03-18T15:38:04.702491+0000","last_became_peered":"2024-03-18T15:38:04.702491+0000","last_unstale":"2024-03-18T15:40:02.770915+0000","last_undegraded":"2024-03-18T15:40:02.770915+0000","last_fullsized":"2024-03-18T15:40:02.770915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:21:37.163989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830028+0000","last_change":"2024-03-18T15:38:03.829112+0000","last_active":"2024-03-18T15:40:02.830028+0000","last_peered":"2024-03-18T15:40:02.830028+0000","last_clean":"2024-03-18T15:40:02.830028+0000","last_became_active":"2024-03-18T15:38:03.828782+0000","last_became_peered":"2024-03-18T15:38:03.828782+0000","last_unstale":"2024-03-18T15:40:02.830028+0000","last_undegraded":"2024-03-18T15:40:02.830028+0000","last_fullsized":"2024-03-18T15:40:02.830028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:53:10.756507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973796+0000","last_change":"2024-03-18T15:38:04.699322+0000","last_active":"2024-03-18T15:40:02.973796+0000","last_peered":"2024-03-18T15:40:02.973796+0000","last_clean":"2024-03-18T15:40:02.973796+0000","last_became_active":"2024-03-18T15:38:04.698963+0000","last_became_peered":"2024-03-18T15:38:04.698963+0000","last_unstale":"2024-03-18T15:40:02.973796+0000","last_undegraded":"2024-03-18T15:40:02.973796+0000","last_fullsized":"2024-03-18T15:40:02.973796+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:13:04.112011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829870+0000","last_change":"2024-03-18T15:38:03.835197+0000","last_active":"2024-03-18T15:40:02.829870+0000","last_peered":"2024-03-18T15:40:02.829870+0000","last_clean":"2024-03-18T15:40:02.829870+0000","last_became_active":"2024-03-18T15:38:03.834828+0000","last_became_peered":"2024-03-18T15:38:03.834828+0000","last_unstale":"2024-03-18T15:40:02.829870+0000","last_undegraded":"2024-03-18T15:40:02.829870+0000","last_fullsized":"2024-03-18T15:40:02.829870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:51.292510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634889+0000","last_change":"2024-03-18T15:39:02.610855+0000","last_active":"2024-03-18T15:40:02.634889+0000","last_peered":"2024-03-18T15:40:02.634889+0000","last_clean":"2024-03-18T15:40:02.634889+0000","last_became_active":"2024-03-18T15:39:02.610513+0000","last_became_peered":"2024-03-18T15:39:02.610513+0000","last_unstale":"2024-03-18T15:40:02.634889+0000","last_undegraded":"2024-03-18T15:40:02.634889+0000","last_fullsized":"2024-03-18T15:40:02.634889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:08:00.284040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:06.269368+0000","last_change":"2024-03-18T15:38:03.829180+0000","last_active":"2024-03-18T15:40:06.269368+0000","last_peered":"2024-03-18T15:40:06.269368+0000","last_clean":"2024-03-18T15:40:06.269368+0000","last_became_active":"2024-03-18T15:38:03.828795+0000","last_became_peered":"2024-03-18T15:38:03.828795+0000","last_unstale":"2024-03-18T15:40:06.269368+0000","last_undegraded":"2024-03-18T15:40:06.269368+0000","last_fullsized":"2024-03-18T15:40:06.269368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:51.015958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829760+0000","last_change":"2024-03-18T15:38:04.690481+0000","last_active":"2024-03-18T15:40:02.829760+0000","last_peered":"2024-03-18T15:40:02.829760+0000","last_clean":"2024-03-18T15:40:02.829760+0000","last_became_active":"2024-03-18T15:38:04.689447+0000","last_became_peered":"2024-03-18T15:38:04.689447+0000","last_unstale":"2024-03-18T15:40:02.829760+0000","last_undegraded":"2024-03-18T15:40:02.829760+0000","last_fullsized":"2024-03-18T15:40:02.829760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:01:19.649312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639528+0000","last_change":"2024-03-18T15:38:03.827432+0000","last_active":"2024-03-18T15:40:02.639528+0000","last_peered":"2024-03-18T15:40:02.639528+0000","last_clean":"2024-03-18T15:40:02.639528+0000","last_became_active":"2024-03-18T15:38:03.827083+0000","last_became_peered":"2024-03-18T15:38:03.827083+0000","last_unstale":"2024-03-18T15:40:02.639528+0000","last_undegraded":"2024-03-18T15:40:02.639528+0000","last_fullsized":"2024-03-18T15:40:02.639528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:28.823831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634737+0000","last_change":"2024-03-18T15:38:04.695452+0000","last_active":"2024-03-18T15:40:02.634737+0000","last_peered":"2024-03-18T15:40:02.634737+0000","last_clean":"2024-03-18T15:40:02.634737+0000","last_became_active":"2024-03-18T15:38:04.695112+0000","last_became_peered":"2024-03-18T15:38:04.695112+0000","last_unstale":"2024-03-18T15:40:02.634737+0000","last_undegraded":"2024-03-18T15:40:02.634737+0000","last_fullsized":"2024-03-18T15:40:02.634737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:26:30.435501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634169+0000","last_change":"2024-03-18T15:38:03.823019+0000","last_active":"2024-03-18T15:40:02.634169+0000","last_peered":"2024-03-18T15:40:02.634169+0000","last_clean":"2024-03-18T15:40:02.634169+0000","last_became_active":"2024-03-18T15:38:03.822653+0000","last_became_peered":"2024-03-18T15:38:03.822653+0000","last_unstale":"2024-03-18T15:40:02.634169+0000","last_undegraded":"2024-03-18T15:40:02.634169+0000","last_fullsized":"2024-03-18T15:40:02.634169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:32:38.484841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638755+0000","last_change":"2024-03-18T15:38:04.685531+0000","last_active":"2024-03-18T15:40:02.638755+0000","last_peered":"2024-03-18T15:40:02.638755+0000","last_clean":"2024-03-18T15:40:02.638755+0000","last_became_active":"2024-03-18T15:38:04.685185+0000","last_became_peered":"2024-03-18T15:38:04.685185+0000","last_unstale":"2024-03-18T15:40:02.638755+0000","last_undegraded":"2024-03-18T15:40:02.638755+0000","last_fullsized":"2024-03-18T15:40:02.638755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:50:58.952015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634379+0000","last_change":"2024-03-18T15:38:03.828941+0000","last_active":"2024-03-18T15:40:02.634379+0000","last_peered":"2024-03-18T15:40:02.634379+0000","last_clean":"2024-03-18T15:40:02.634379+0000","last_became_active":"2024-03-18T15:38:03.827070+0000","last_became_peered":"2024-03-18T15:38:03.827070+0000","last_unstale":"2024-03-18T15:40:02.634379+0000","last_undegraded":"2024-03-18T15:40:02.634379+0000","last_fullsized":"2024-03-18T15:40:02.634379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:40:50.572092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827833+0000","last_change":"2024-03-18T15:38:04.683647+0000","last_active":"2024-03-18T15:40:02.827833+0000","last_peered":"2024-03-18T15:40:02.827833+0000","last_clean":"2024-03-18T15:40:02.827833+0000","last_became_active":"2024-03-18T15:38:04.683311+0000","last_became_peered":"2024-03-18T15:38:04.683311+0000","last_unstale":"2024-03-18T15:40:02.827833+0000","last_undegraded":"2024-03-18T15:40:02.827833+0000","last_fullsized":"2024-03-18T15:40:02.827833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:54:15.583797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":268,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:10.272934+0000","last_change":"2024-03-18T15:38:03.824083+0000","last_active":"2024-03-18T15:40:10.272934+0000","last_peered":"2024-03-18T15:40:10.272934+0000","last_clean":"2024-03-18T15:40:10.272934+0000","last_became_active":"2024-03-18T15:38:03.823697+0000","last_became_peered":"2024-03-18T15:38:03.823697+0000","last_unstale":"2024-03-18T15:40:10.272934+0000","last_undegraded":"2024-03-18T15:40:10.272934+0000","last_fullsized":"2024-03-18T15:40:10.272934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:57:09.095839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.974209+0000","last_change":"2024-03-18T15:38:04.698879+0000","last_active":"2024-03-18T15:40:02.974209+0000","last_peered":"2024-03-18T15:40:02.974209+0000","last_clean":"2024-03-18T15:40:02.974209+0000","last_became_active":"2024-03-18T15:38:04.698375+0000","last_became_peered":"2024-03-18T15:38:04.698375+0000","last_unstale":"2024-03-18T15:40:02.974209+0000","last_undegraded":"2024-03-18T15:40:02.974209+0000","last_fullsized":"2024-03-18T15:40:02.974209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:47:21.931811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638900+0000","last_change":"2024-03-18T15:38:03.830620+0000","last_active":"2024-03-18T15:40:02.638900+0000","last_peered":"2024-03-18T15:40:02.638900+0000","last_clean":"2024-03-18T15:40:02.638900+0000","last_became_active":"2024-03-18T15:38:03.830264+0000","last_became_peered":"2024-03-18T15:38:03.830264+0000","last_unstale":"2024-03-18T15:40:02.638900+0000","last_undegraded":"2024-03-18T15:40:02.638900+0000","last_fullsized":"2024-03-18T15:40:02.638900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:28:49.032044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770819+0000","last_change":"2024-03-18T15:39:02.611014+0000","last_active":"2024-03-18T15:40:02.770819+0000","last_peered":"2024-03-18T15:40:02.770819+0000","last_clean":"2024-03-18T15:40:02.770819+0000","last_became_active":"2024-03-18T15:39:02.610574+0000","last_became_peered":"2024-03-18T15:39:02.610574+0000","last_unstale":"2024-03-18T15:40:02.770819+0000","last_undegraded":"2024-03-18T15:40:02.770819+0000","last_fullsized":"2024-03-18T15:40:02.770819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:00:40.612478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634921+0000","last_change":"2024-03-18T15:38:03.829664+0000","last_active":"2024-03-18T15:40:02.634921+0000","last_peered":"2024-03-18T15:40:02.634921+0000","last_clean":"2024-03-18T15:40:02.634921+0000","last_became_active":"2024-03-18T15:38:03.829135+0000","last_became_peered":"2024-03-18T15:38:03.829135+0000","last_unstale":"2024-03-18T15:40:02.634921+0000","last_undegraded":"2024-03-18T15:40:02.634921+0000","last_fullsized":"2024-03-18T15:40:02.634921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:52:53.771606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973967+0000","last_change":"2024-03-18T15:38:04.702506+0000","last_active":"2024-03-18T15:40:02.973967+0000","last_peered":"2024-03-18T15:40:02.973967+0000","last_clean":"2024-03-18T15:40:02.973967+0000","last_became_active":"2024-03-18T15:38:04.702058+0000","last_became_peered":"2024-03-18T15:38:04.702058+0000","last_unstale":"2024-03-18T15:40:02.973967+0000","last_undegraded":"2024-03-18T15:40:02.973967+0000","last_fullsized":"2024-03-18T15:40:02.973967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:03.506118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":310,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:10.273239+0000","last_change":"2024-03-18T15:38:03.829137+0000","last_active":"2024-03-18T15:40:10.273239+0000","last_peered":"2024-03-18T15:40:10.273239+0000","last_clean":"2024-03-18T15:40:10.273239+0000","last_became_active":"2024-03-18T15:38:03.828661+0000","last_became_peered":"2024-03-18T15:38:03.828661+0000","last_unstale":"2024-03-18T15:40:10.273239+0000","last_undegraded":"2024-03-18T15:40:10.273239+0000","last_fullsized":"2024-03-18T15:40:10.273239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:01:45.168200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69887,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":194,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829918+0000","last_change":"2024-03-18T15:38:04.693624+0000","last_active":"2024-03-18T15:40:02.829918+0000","last_peered":"2024-03-18T15:40:02.829918+0000","last_clean":"2024-03-18T15:40:02.829918+0000","last_became_active":"2024-03-18T15:38:04.693174+0000","last_became_peered":"2024-03-18T15:38:04.693174+0000","last_unstale":"2024-03-18T15:40:02.829918+0000","last_undegraded":"2024-03-18T15:40:02.829918+0000","last_fullsized":"2024-03-18T15:40:02.829918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:40:00.610479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.649987+0000","last_change":"2024-03-18T15:40:02.649987+0000","last_active":"2024-03-18T15:40:02.649987+0000","last_peered":"2024-03-18T15:40:02.649987+0000","last_clean":"2024-03-18T15:40:02.649987+0000","last_became_active":"2024-03-18T15:40:02.640566+0000","last_became_peered":"2024-03-18T15:40:02.640566+0000","last_unstale":"2024-03-18T15:40:02.649987+0000","last_undegraded":"2024-03-18T15:40:02.649987+0000","last_fullsized":"2024-03-18T15:40:02.649987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:49:31.828795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634164+0000","last_change":"2024-03-18T15:38:04.689526+0000","last_active":"2024-03-18T15:40:02.634164+0000","last_peered":"2024-03-18T15:40:02.634164+0000","last_clean":"2024-03-18T15:40:02.634164+0000","last_became_active":"2024-03-18T15:38:04.689213+0000","last_became_peered":"2024-03-18T15:38:04.689213+0000","last_unstale":"2024-03-18T15:40:02.634164+0000","last_undegraded":"2024-03-18T15:40:02.634164+0000","last_fullsized":"2024-03-18T15:40:02.634164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:09:16.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634628+0000","last_change":"2024-03-18T15:38:03.826604+0000","last_active":"2024-03-18T15:40:02.634628+0000","last_peered":"2024-03-18T15:40:02.634628+0000","last_clean":"2024-03-18T15:40:02.634628+0000","last_became_active":"2024-03-18T15:38:03.825459+0000","last_became_peered":"2024-03-18T15:38:03.825459+0000","last_unstale":"2024-03-18T15:40:02.634628+0000","last_undegraded":"2024-03-18T15:40:02.634628+0000","last_fullsized":"2024-03-18T15:40:02.634628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:58:25.242381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639104+0000","last_change":"2024-03-18T15:38:04.698679+0000","last_active":"2024-03-18T15:40:02.639104+0000","last_peered":"2024-03-18T15:40:02.639104+0000","last_clean":"2024-03-18T15:40:02.639104+0000","last_became_active":"2024-03-18T15:38:04.698330+0000","last_became_peered":"2024-03-18T15:38:04.698330+0000","last_unstale":"2024-03-18T15:40:02.639104+0000","last_undegraded":"2024-03-18T15:40:02.639104+0000","last_fullsized":"2024-03-18T15:40:02.639104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:34:22.090197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:07.275296+0000","last_change":"2024-03-18T15:38:03.828501+0000","last_active":"2024-03-18T15:40:07.275296+0000","last_peered":"2024-03-18T15:40:07.275296+0000","last_clean":"2024-03-18T15:40:07.275296+0000","last_became_active":"2024-03-18T15:38:03.828116+0000","last_became_peered":"2024-03-18T15:38:03.828116+0000","last_unstale":"2024-03-18T15:40:07.275296+0000","last_undegraded":"2024-03-18T15:40:07.275296+0000","last_fullsized":"2024-03-18T15:40:07.275296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:05.192900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638734+0000","last_change":"2024-03-18T15:38:04.695968+0000","last_active":"2024-03-18T15:40:02.638734+0000","last_peered":"2024-03-18T15:40:02.638734+0000","last_clean":"2024-03-18T15:40:02.638734+0000","last_became_active":"2024-03-18T15:38:04.695633+0000","last_became_peered":"2024-03-18T15:38:04.695633+0000","last_unstale":"2024-03-18T15:40:02.638734+0000","last_undegraded":"2024-03-18T15:40:02.638734+0000","last_fullsized":"2024-03-18T15:40:02.638734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:30:04.246527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829600+0000","last_change":"2024-03-18T15:38:03.833904+0000","last_active":"2024-03-18T15:40:02.829600+0000","last_peered":"2024-03-18T15:40:02.829600+0000","last_clean":"2024-03-18T15:40:02.829600+0000","last_became_active":"2024-03-18T15:38:03.833516+0000","last_became_peered":"2024-03-18T15:38:03.833516+0000","last_unstale":"2024-03-18T15:40:02.829600+0000","last_undegraded":"2024-03-18T15:40:02.829600+0000","last_fullsized":"2024-03-18T15:40:02.829600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:02:03.913843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634437+0000","last_change":"2024-03-18T15:38:04.702896+0000","last_active":"2024-03-18T15:40:02.634437+0000","last_peered":"2024-03-18T15:40:02.634437+0000","last_clean":"2024-03-18T15:40:02.634437+0000","last_became_active":"2024-03-18T15:38:04.702573+0000","last_became_peered":"2024-03-18T15:38:04.702573+0000","last_unstale":"2024-03-18T15:40:02.634437+0000","last_undegraded":"2024-03-18T15:40:02.634437+0000","last_fullsized":"2024-03-18T15:40:02.634437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:21:38.678914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637233+0000","last_change":"2024-03-18T15:39:02.578097+0000","last_active":"2024-03-18T15:40:02.637233+0000","last_peered":"2024-03-18T15:40:02.637233+0000","last_clean":"2024-03-18T15:40:02.637233+0000","last_became_active":"2024-03-18T15:39:02.572158+0000","last_became_peered":"2024-03-18T15:39:02.572158+0000","last_unstale":"2024-03-18T15:40:02.637233+0000","last_undegraded":"2024-03-18T15:40:02.637233+0000","last_fullsized":"2024-03-18T15:40:02.637233+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:13:51.775922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637255+0000","last_change":"2024-03-18T15:39:02.571921+0000","last_active":"2024-03-18T15:40:02.637255+0000","last_peered":"2024-03-18T15:40:02.637255+0000","last_clean":"2024-03-18T15:40:02.637255+0000","last_became_active":"2024-03-18T15:39:02.571555+0000","last_became_peered":"2024-03-18T15:39:02.571555+0000","last_unstale":"2024-03-18T15:40:02.637255+0000","last_undegraded":"2024-03-18T15:40:02.637255+0000","last_fullsized":"2024-03-18T15:40:02.637255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:02:31.840980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.827933+0000","last_change":"2024-03-18T15:38:03.829619+0000","last_active":"2024-03-18T15:40:02.827933+0000","last_peered":"2024-03-18T15:40:02.827933+0000","last_clean":"2024-03-18T15:40:02.827933+0000","last_became_active":"2024-03-18T15:38:03.829295+0000","last_became_peered":"2024-03-18T15:38:03.829295+0000","last_unstale":"2024-03-18T15:40:02.827933+0000","last_undegraded":"2024-03-18T15:40:02.827933+0000","last_fullsized":"2024-03-18T15:40:02.827933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:17:14.582652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639726+0000","last_change":"2024-03-18T15:38:04.685164+0000","last_active":"2024-03-18T15:40:02.639726+0000","last_peered":"2024-03-18T15:40:02.639726+0000","last_clean":"2024-03-18T15:40:02.639726+0000","last_became_active":"2024-03-18T15:38:04.684849+0000","last_became_peered":"2024-03-18T15:38:04.684849+0000","last_unstale":"2024-03-18T15:40:02.639726+0000","last_undegraded":"2024-03-18T15:40:02.639726+0000","last_fullsized":"2024-03-18T15:40:02.639726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:53.283031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:08.281001+0000","last_change":"2024-03-18T15:38:03.832941+0000","last_active":"2024-03-18T15:40:08.281001+0000","last_peered":"2024-03-18T15:40:08.281001+0000","last_clean":"2024-03-18T15:40:08.281001+0000","last_became_active":"2024-03-18T15:38:03.832545+0000","last_became_peered":"2024-03-18T15:38:03.832545+0000","last_unstale":"2024-03-18T15:40:08.281001+0000","last_undegraded":"2024-03-18T15:40:08.281001+0000","last_fullsized":"2024-03-18T15:40:08.281001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:11:08.036217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639799+0000","last_change":"2024-03-18T15:38:04.689219+0000","last_active":"2024-03-18T15:40:02.639799+0000","last_peered":"2024-03-18T15:40:02.639799+0000","last_clean":"2024-03-18T15:40:02.639799+0000","last_became_active":"2024-03-18T15:38:04.688800+0000","last_became_peered":"2024-03-18T15:38:04.688800+0000","last_unstale":"2024-03-18T15:40:02.639799+0000","last_undegraded":"2024-03-18T15:40:02.639799+0000","last_fullsized":"2024-03-18T15:40:02.639799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:56:35.273798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634550+0000","last_change":"2024-03-18T15:38:04.695657+0000","last_active":"2024-03-18T15:40:02.634550+0000","last_peered":"2024-03-18T15:40:02.634550+0000","last_clean":"2024-03-18T15:40:02.634550+0000","last_became_active":"2024-03-18T15:38:04.695311+0000","last_became_peered":"2024-03-18T15:38:04.695311+0000","last_unstale":"2024-03-18T15:40:02.634550+0000","last_undegraded":"2024-03-18T15:40:02.634550+0000","last_fullsized":"2024-03-18T15:40:02.634550+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:14.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:06.273966+0000","last_change":"2024-03-18T15:38:03.827499+0000","last_active":"2024-03-18T15:40:06.273966+0000","last_peered":"2024-03-18T15:40:06.273966+0000","last_clean":"2024-03-18T15:40:06.273966+0000","last_became_active":"2024-03-18T15:38:03.827144+0000","last_became_peered":"2024-03-18T15:38:03.827144+0000","last_unstale":"2024-03-18T15:40:06.273966+0000","last_undegraded":"2024-03-18T15:40:06.273966+0000","last_fullsized":"2024-03-18T15:40:06.273966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:46:35.776926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638875+0000","last_change":"2024-03-18T15:39:02.610861+0000","last_active":"2024-03-18T15:40:02.638875+0000","last_peered":"2024-03-18T15:40:02.638875+0000","last_clean":"2024-03-18T15:40:02.638875+0000","last_became_active":"2024-03-18T15:39:02.610459+0000","last_became_peered":"2024-03-18T15:39:02.610459+0000","last_unstale":"2024-03-18T15:40:02.638875+0000","last_undegraded":"2024-03-18T15:40:02.638875+0000","last_fullsized":"2024-03-18T15:40:02.638875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:20:20.555904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973870+0000","last_change":"2024-03-18T15:38:03.826911+0000","last_active":"2024-03-18T15:40:02.973870+0000","last_peered":"2024-03-18T15:40:02.973870+0000","last_clean":"2024-03-18T15:40:02.973870+0000","last_became_active":"2024-03-18T15:38:03.826533+0000","last_became_peered":"2024-03-18T15:38:03.826533+0000","last_unstale":"2024-03-18T15:40:02.973870+0000","last_undegraded":"2024-03-18T15:40:02.973870+0000","last_fullsized":"2024-03-18T15:40:02.973870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:42:52.081706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638820+0000","last_change":"2024-03-18T15:36:49.134432+0000","last_active":"2024-03-18T15:40:02.638820+0000","last_peered":"2024-03-18T15:40:02.638820+0000","last_clean":"2024-03-18T15:40:02.638820+0000","last_became_active":"2024-03-18T15:36:49.133988+0000","last_became_peered":"2024-03-18T15:36:49.133988+0000","last_unstale":"2024-03-18T15:40:02.638820+0000","last_undegraded":"2024-03-18T15:40:02.638820+0000","last_fullsized":"2024-03-18T15:40:02.638820+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:23.849692+0000","last_clean_scrub_stamp":"2024-03-18T15:33:23.849692+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:17.592419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634027+0000","last_change":"2024-03-18T15:38:04.693607+0000","last_active":"2024-03-18T15:40:02.634027+0000","last_peered":"2024-03-18T15:40:02.634027+0000","last_clean":"2024-03-18T15:40:02.634027+0000","last_became_active":"2024-03-18T15:38:04.693275+0000","last_became_peered":"2024-03-18T15:38:04.693275+0000","last_unstale":"2024-03-18T15:40:02.634027+0000","last_undegraded":"2024-03-18T15:40:02.634027+0000","last_fullsized":"2024-03-18T15:40:02.634027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:05:49.723971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638961+0000","last_change":"2024-03-18T15:38:03.832733+0000","last_active":"2024-03-18T15:40:02.638961+0000","last_peered":"2024-03-18T15:40:02.638961+0000","last_clean":"2024-03-18T15:40:02.638961+0000","last_became_active":"2024-03-18T15:38:03.831699+0000","last_became_peered":"2024-03-18T15:38:03.831699+0000","last_unstale":"2024-03-18T15:40:02.638961+0000","last_undegraded":"2024-03-18T15:40:02.638961+0000","last_fullsized":"2024-03-18T15:40:02.638961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:59:26.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.830021+0000","last_change":"2024-03-18T15:38:03.826359+0000","last_active":"2024-03-18T15:40:02.830021+0000","last_peered":"2024-03-18T15:40:02.830021+0000","last_clean":"2024-03-18T15:40:02.830021+0000","last_became_active":"2024-03-18T15:38:03.826020+0000","last_became_peered":"2024-03-18T15:38:03.826020+0000","last_unstale":"2024-03-18T15:40:02.830021+0000","last_undegraded":"2024-03-18T15:40:02.830021+0000","last_fullsized":"2024-03-18T15:40:02.830021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:52:46.572967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634273+0000","last_change":"2024-03-18T15:38:04.678847+0000","last_active":"2024-03-18T15:40:02.634273+0000","last_peered":"2024-03-18T15:40:02.634273+0000","last_clean":"2024-03-18T15:40:02.634273+0000","last_became_active":"2024-03-18T15:38:04.678412+0000","last_became_peered":"2024-03-18T15:38:04.678412+0000","last_unstale":"2024-03-18T15:40:02.634273+0000","last_undegraded":"2024-03-18T15:40:02.634273+0000","last_fullsized":"2024-03-18T15:40:02.634273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:12.700256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829178+0000","last_change":"2024-03-18T15:38:03.824168+0000","last_active":"2024-03-18T15:40:02.829178+0000","last_peered":"2024-03-18T15:40:02.829178+0000","last_clean":"2024-03-18T15:40:02.829178+0000","last_became_active":"2024-03-18T15:38:03.823822+0000","last_became_peered":"2024-03-18T15:38:03.823822+0000","last_unstale":"2024-03-18T15:40:02.829178+0000","last_undegraded":"2024-03-18T15:40:02.829178+0000","last_fullsized":"2024-03-18T15:40:02.829178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:43:33.455250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639056+0000","last_change":"2024-03-18T15:38:04.691264+0000","last_active":"2024-03-18T15:40:02.639056+0000","last_peered":"2024-03-18T15:40:02.639056+0000","last_clean":"2024-03-18T15:40:02.639056+0000","last_became_active":"2024-03-18T15:38:04.690938+0000","last_became_peered":"2024-03-18T15:38:04.690938+0000","last_unstale":"2024-03-18T15:40:02.639056+0000","last_undegraded":"2024-03-18T15:40:02.639056+0000","last_fullsized":"2024-03-18T15:40:02.639056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:07:33.492764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973914+0000","last_change":"2024-03-18T15:38:03.828314+0000","last_active":"2024-03-18T15:40:02.973914+0000","last_peered":"2024-03-18T15:40:02.973914+0000","last_clean":"2024-03-18T15:40:02.973914+0000","last_became_active":"2024-03-18T15:38:03.827896+0000","last_became_peered":"2024-03-18T15:38:03.827896+0000","last_unstale":"2024-03-18T15:40:02.973914+0000","last_undegraded":"2024-03-18T15:40:02.973914+0000","last_fullsized":"2024-03-18T15:40:02.973914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:48:17.343434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770696+0000","last_change":"2024-03-18T15:38:04.689274+0000","last_active":"2024-03-18T15:40:02.770696+0000","last_peered":"2024-03-18T15:40:02.770696+0000","last_clean":"2024-03-18T15:40:02.770696+0000","last_became_active":"2024-03-18T15:38:04.688769+0000","last_became_peered":"2024-03-18T15:38:04.688769+0000","last_unstale":"2024-03-18T15:40:02.770696+0000","last_undegraded":"2024-03-18T15:40:02.770696+0000","last_fullsized":"2024-03-18T15:40:02.770696+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:04:58.659008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973437+0000","last_change":"2024-03-18T15:38:03.830673+0000","last_active":"2024-03-18T15:40:02.973437+0000","last_peered":"2024-03-18T15:40:02.973437+0000","last_clean":"2024-03-18T15:40:02.973437+0000","last_became_active":"2024-03-18T15:38:03.830039+0000","last_became_peered":"2024-03-18T15:38:03.830039+0000","last_unstale":"2024-03-18T15:40:02.973437+0000","last_undegraded":"2024-03-18T15:40:02.973437+0000","last_fullsized":"2024-03-18T15:40:02.973437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:44:23.900081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829589+0000","last_change":"2024-03-18T15:38:04.697265+0000","last_active":"2024-03-18T15:40:02.829589+0000","last_peered":"2024-03-18T15:40:02.829589+0000","last_clean":"2024-03-18T15:40:02.829589+0000","last_became_active":"2024-03-18T15:38:04.696927+0000","last_became_peered":"2024-03-18T15:38:04.696927+0000","last_unstale":"2024-03-18T15:40:02.829589+0000","last_undegraded":"2024-03-18T15:40:02.829589+0000","last_fullsized":"2024-03-18T15:40:02.829589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638847+0000","last_change":"2024-03-18T15:38:03.832449+0000","last_active":"2024-03-18T15:40:02.638847+0000","last_peered":"2024-03-18T15:40:02.638847+0000","last_clean":"2024-03-18T15:40:02.638847+0000","last_became_active":"2024-03-18T15:38:03.832077+0000","last_became_peered":"2024-03-18T15:38:03.832077+0000","last_unstale":"2024-03-18T15:40:02.638847+0000","last_undegraded":"2024-03-18T15:40:02.638847+0000","last_fullsized":"2024-03-18T15:40:02.638847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:47:49.737391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637056+0000","last_change":"2024-03-18T15:38:04.694348+0000","last_active":"2024-03-18T15:40:02.637056+0000","last_peered":"2024-03-18T15:40:02.637056+0000","last_clean":"2024-03-18T15:40:02.637056+0000","last_became_active":"2024-03-18T15:38:04.694001+0000","last_became_peered":"2024-03-18T15:38:04.694001+0000","last_unstale":"2024-03-18T15:40:02.637056+0000","last_undegraded":"2024-03-18T15:40:02.637056+0000","last_fullsized":"2024-03-18T15:40:02.637056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:29.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639128+0000","last_change":"2024-03-18T15:38:03.834599+0000","last_active":"2024-03-18T15:40:02.639128+0000","last_peered":"2024-03-18T15:40:02.639128+0000","last_clean":"2024-03-18T15:40:02.639128+0000","last_became_active":"2024-03-18T15:38:03.833399+0000","last_became_peered":"2024-03-18T15:38:03.833399+0000","last_unstale":"2024-03-18T15:40:02.639128+0000","last_undegraded":"2024-03-18T15:40:02.639128+0000","last_fullsized":"2024-03-18T15:40:02.639128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:24:39.400115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638975+0000","last_change":"2024-03-18T15:38:04.689275+0000","last_active":"2024-03-18T15:40:02.638975+0000","last_peered":"2024-03-18T15:40:02.638975+0000","last_clean":"2024-03-18T15:40:02.638975+0000","last_became_active":"2024-03-18T15:38:04.688862+0000","last_became_peered":"2024-03-18T15:38:04.688862+0000","last_unstale":"2024-03-18T15:40:02.638975+0000","last_undegraded":"2024-03-18T15:40:02.638975+0000","last_fullsized":"2024-03-18T15:40:02.638975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:41:01.838402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.829711+0000","last_change":"2024-03-18T15:39:02.581338+0000","last_active":"2024-03-18T15:40:02.829711+0000","last_peered":"2024-03-18T15:40:02.829711+0000","last_clean":"2024-03-18T15:40:02.829711+0000","last_became_active":"2024-03-18T15:39:02.573270+0000","last_became_peered":"2024-03-18T15:39:02.573270+0000","last_unstale":"2024-03-18T15:40:02.829711+0000","last_undegraded":"2024-03-18T15:40:02.829711+0000","last_fullsized":"2024-03-18T15:40:02.829711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:07:43.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639316+0000","last_change":"2024-03-18T15:38:04.698621+0000","last_active":"2024-03-18T15:40:02.639316+0000","last_peered":"2024-03-18T15:40:02.639316+0000","last_clean":"2024-03-18T15:40:02.639316+0000","last_became_active":"2024-03-18T15:38:04.698198+0000","last_became_peered":"2024-03-18T15:38:04.698198+0000","last_unstale":"2024-03-18T15:40:02.639316+0000","last_undegraded":"2024-03-18T15:40:02.639316+0000","last_fullsized":"2024-03-18T15:40:02.639316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:26:54.518838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:07.289529+0000","last_change":"2024-03-18T15:38:03.836836+0000","last_active":"2024-03-18T15:40:07.289529+0000","last_peered":"2024-03-18T15:40:07.289529+0000","last_clean":"2024-03-18T15:40:07.289529+0000","last_became_active":"2024-03-18T15:38:03.836501+0000","last_became_peered":"2024-03-18T15:38:03.836501+0000","last_unstale":"2024-03-18T15:40:07.289529+0000","last_undegraded":"2024-03-18T15:40:07.289529+0000","last_fullsized":"2024-03-18T15:40:07.289529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:06:07.901080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639069+0000","last_change":"2024-03-18T15:38:04.697973+0000","last_active":"2024-03-18T15:40:02.639069+0000","last_peered":"2024-03-18T15:40:02.639069+0000","last_clean":"2024-03-18T15:40:02.639069+0000","last_became_active":"2024-03-18T15:38:04.697328+0000","last_became_peered":"2024-03-18T15:38:04.697328+0000","last_unstale":"2024-03-18T15:40:02.639069+0000","last_undegraded":"2024-03-18T15:40:02.639069+0000","last_fullsized":"2024-03-18T15:40:02.639069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:11:06.595405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634814+0000","last_change":"2024-03-18T15:38:03.826364+0000","last_active":"2024-03-18T15:40:02.634814+0000","last_peered":"2024-03-18T15:40:02.634814+0000","last_clean":"2024-03-18T15:40:02.634814+0000","last_became_active":"2024-03-18T15:38:03.825993+0000","last_became_peered":"2024-03-18T15:38:03.825993+0000","last_unstale":"2024-03-18T15:40:02.634814+0000","last_undegraded":"2024-03-18T15:40:02.634814+0000","last_fullsized":"2024-03-18T15:40:02.634814+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:03.565063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638984+0000","last_change":"2024-03-18T15:38:04.695066+0000","last_active":"2024-03-18T15:40:02.638984+0000","last_peered":"2024-03-18T15:40:02.638984+0000","last_clean":"2024-03-18T15:40:02.638984+0000","last_became_active":"2024-03-18T15:38:04.694204+0000","last_became_peered":"2024-03-18T15:38:04.694204+0000","last_unstale":"2024-03-18T15:40:02.638984+0000","last_undegraded":"2024-03-18T15:40:02.638984+0000","last_fullsized":"2024-03-18T15:40:02.638984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:41:01.574959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634335+0000","last_change":"2024-03-18T15:38:03.829874+0000","last_active":"2024-03-18T15:40:02.634335+0000","last_peered":"2024-03-18T15:40:02.634335+0000","last_clean":"2024-03-18T15:40:02.634335+0000","last_became_active":"2024-03-18T15:38:03.829543+0000","last_became_peered":"2024-03-18T15:38:03.829543+0000","last_unstale":"2024-03-18T15:40:02.634335+0000","last_undegraded":"2024-03-18T15:40:02.634335+0000","last_fullsized":"2024-03-18T15:40:02.634335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:02.804292+0000","last_clean_scrub_stamp":"2024-03-18T15:38:02.804292+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:21:33.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639024+0000","last_change":"2024-03-18T15:38:04.695274+0000","last_active":"2024-03-18T15:40:02.639024+0000","last_peered":"2024-03-18T15:40:02.639024+0000","last_clean":"2024-03-18T15:40:02.639024+0000","last_became_active":"2024-03-18T15:38:04.694493+0000","last_became_peered":"2024-03-18T15:38:04.694493+0000","last_unstale":"2024-03-18T15:40:02.639024+0000","last_undegraded":"2024-03-18T15:40:02.639024+0000","last_fullsized":"2024-03-18T15:40:02.639024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:25:06.854217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637119+0000","last_change":"2024-03-18T15:38:04.694385+0000","last_active":"2024-03-18T15:40:02.637119+0000","last_peered":"2024-03-18T15:40:02.637119+0000","last_clean":"2024-03-18T15:40:02.637119+0000","last_became_active":"2024-03-18T15:38:04.693076+0000","last_became_peered":"2024-03-18T15:38:04.693076+0000","last_unstale":"2024-03-18T15:40:02.637119+0000","last_undegraded":"2024-03-18T15:40:02.637119+0000","last_fullsized":"2024-03-18T15:40:02.637119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:41:11.214651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638939+0000","last_change":"2024-03-18T15:38:04.695977+0000","last_active":"2024-03-18T15:40:02.638939+0000","last_peered":"2024-03-18T15:40:02.638939+0000","last_clean":"2024-03-18T15:40:02.638939+0000","last_became_active":"2024-03-18T15:38:04.695085+0000","last_became_peered":"2024-03-18T15:38:04.695085+0000","last_unstale":"2024-03-18T15:40:02.638939+0000","last_undegraded":"2024-03-18T15:40:02.638939+0000","last_fullsized":"2024-03-18T15:40:02.638939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:01:09.125335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638904+0000","last_change":"2024-03-18T15:39:02.574194+0000","last_active":"2024-03-18T15:40:02.638904+0000","last_peered":"2024-03-18T15:40:02.638904+0000","last_clean":"2024-03-18T15:40:02.638904+0000","last_became_active":"2024-03-18T15:39:02.573841+0000","last_became_peered":"2024-03-18T15:39:02.573841+0000","last_unstale":"2024-03-18T15:40:02.638904+0000","last_undegraded":"2024-03-18T15:40:02.638904+0000","last_fullsized":"2024-03-18T15:40:02.638904+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:41.866521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,0],"acting":[2,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.770759+0000","last_change":"2024-03-18T15:38:04.692984+0000","last_active":"2024-03-18T15:40:02.770759+0000","last_peered":"2024-03-18T15:40:02.770759+0000","last_clean":"2024-03-18T15:40:02.770759+0000","last_became_active":"2024-03-18T15:38:04.692650+0000","last_became_peered":"2024-03-18T15:38:04.692650+0000","last_unstale":"2024-03-18T15:40:02.770759+0000","last_undegraded":"2024-03-18T15:40:02.770759+0000","last_fullsized":"2024-03-18T15:40:02.770759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:35:56.059024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634266+0000","last_change":"2024-03-18T15:38:04.699482+0000","last_active":"2024-03-18T15:40:02.634266+0000","last_peered":"2024-03-18T15:40:02.634266+0000","last_clean":"2024-03-18T15:40:02.634266+0000","last_became_active":"2024-03-18T15:38:04.699166+0000","last_became_peered":"2024-03-18T15:38:04.699166+0000","last_unstale":"2024-03-18T15:40:02.634266+0000","last_undegraded":"2024-03-18T15:40:02.634266+0000","last_fullsized":"2024-03-18T15:40:02.634266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:32:00.474072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637235+0000","last_change":"2024-03-18T15:38:04.692382+0000","last_active":"2024-03-18T15:40:02.637235+0000","last_peered":"2024-03-18T15:40:02.637235+0000","last_clean":"2024-03-18T15:40:02.637235+0000","last_became_active":"2024-03-18T15:38:04.692015+0000","last_became_peered":"2024-03-18T15:38:04.692015+0000","last_unstale":"2024-03-18T15:40:02.637235+0000","last_undegraded":"2024-03-18T15:40:02.637235+0000","last_fullsized":"2024-03-18T15:40:02.637235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:48:00.828860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639384+0000","last_change":"2024-03-18T15:38:04.692119+0000","last_active":"2024-03-18T15:40:02.639384+0000","last_peered":"2024-03-18T15:40:02.639384+0000","last_clean":"2024-03-18T15:40:02.639384+0000","last_became_active":"2024-03-18T15:38:04.691647+0000","last_became_peered":"2024-03-18T15:38:04.691647+0000","last_unstale":"2024-03-18T15:40:02.639384+0000","last_undegraded":"2024-03-18T15:40:02.639384+0000","last_fullsized":"2024-03-18T15:40:02.639384+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:16:06.545513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.639014+0000","last_change":"2024-03-18T15:38:04.699624+0000","last_active":"2024-03-18T15:40:02.639014+0000","last_peered":"2024-03-18T15:40:02.639014+0000","last_clean":"2024-03-18T15:40:02.639014+0000","last_became_active":"2024-03-18T15:38:04.699298+0000","last_became_peered":"2024-03-18T15:38:04.699298+0000","last_unstale":"2024-03-18T15:40:02.639014+0000","last_undegraded":"2024-03-18T15:40:02.639014+0000","last_fullsized":"2024-03-18T15:40:02.639014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:39:13.919349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634448+0000","last_change":"2024-03-18T15:38:04.694781+0000","last_active":"2024-03-18T15:40:02.634448+0000","last_peered":"2024-03-18T15:40:02.634448+0000","last_clean":"2024-03-18T15:40:02.634448+0000","last_became_active":"2024-03-18T15:38:04.692348+0000","last_became_peered":"2024-03-18T15:38:04.692348+0000","last_unstale":"2024-03-18T15:40:02.634448+0000","last_undegraded":"2024-03-18T15:40:02.634448+0000","last_fullsized":"2024-03-18T15:40:02.634448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:27.833281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.638711+0000","last_change":"2024-03-18T15:38:04.690263+0000","last_active":"2024-03-18T15:40:02.638711+0000","last_peered":"2024-03-18T15:40:02.638711+0000","last_clean":"2024-03-18T15:40:02.638711+0000","last_became_active":"2024-03-18T15:38:04.689943+0000","last_became_peered":"2024-03-18T15:38:04.689943+0000","last_unstale":"2024-03-18T15:40:02.638711+0000","last_undegraded":"2024-03-18T15:40:02.638711+0000","last_fullsized":"2024-03-18T15:40:02.638711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:53:57.762128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.973537+0000","last_change":"2024-03-18T15:38:04.696013+0000","last_active":"2024-03-18T15:40:02.973537+0000","last_peered":"2024-03-18T15:40:02.973537+0000","last_clean":"2024-03-18T15:40:02.973537+0000","last_became_active":"2024-03-18T15:38:04.695649+0000","last_became_peered":"2024-03-18T15:38:04.695649+0000","last_unstale":"2024-03-18T15:40:02.973537+0000","last_undegraded":"2024-03-18T15:40:02.973537+0000","last_fullsized":"2024-03-18T15:40:02.973537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:17:52.562057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.634382+0000","last_change":"2024-03-18T15:38:04.689467+0000","last_active":"2024-03-18T15:40:02.634382+0000","last_peered":"2024-03-18T15:40:02.634382+0000","last_clean":"2024-03-18T15:40:02.634382+0000","last_became_active":"2024-03-18T15:38:04.688276+0000","last_became_peered":"2024-03-18T15:38:04.688276+0000","last_unstale":"2024-03-18T15:40:02.634382+0000","last_undegraded":"2024-03-18T15:40:02.634382+0000","last_fullsized":"2024-03-18T15:40:02.634382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:58:46.184512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-18T15:40:02.637251+0000","last_change":"2024-03-18T15:39:02.569338+0000","last_active":"2024-03-18T15:40:02.637251+0000","last_peered":"2024-03-18T15:40:02.637251+0000","last_clean":"2024-03-18T15:40:02.637251+0000","last_became_active":"2024-03-18T15:39:02.568989+0000","last_became_peered":"2024-03-18T15:39:02.568989+0000","last_unstale":"2024-03-18T15:40:02.637251+0000","last_undegraded":"2024-03-18T15:40:02.637251+0000","last_fullsized":"2024-03-18T15:40:02.637251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:38:03.806767+0000","last_clean_scrub_stamp":"2024-03-18T15:38:03.806767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:01:00.069405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":245895,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77008,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1228,"num_read_kb":701,"num_write":254,"num_write_kb":152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":90,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":231024,"data_compressed":5604,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":478900,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1139643,"data_compressed":49176,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":478349,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":478705,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1071162,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462977,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":478066,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":548866,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":479796,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":549409,"data_compressed":13579,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":477891,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":478235,"data_compressed":11711,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1070532,"data_compressed":47308,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33955,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35597,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71639,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":72182,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":71931,"data_compressed":1868,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:40:12.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.312+0000 7f9a8c2de640 1 -- 172.21.15.96:0/1820601083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 msgr2=0x7f9a84101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.312+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/1820601083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84101d70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009900 tx=0x7f9a7402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- 172.21.15.96:0/1820601083 shutdown_connections 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/1820601083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a841022b0 0x7f9a84108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/1820601083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84101d70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/1820601083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a84100700 0x7f9a84100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- 172.21.15.96:0/1820601083 >> 172.21.15.96:0/1820601083 conn(0x7f9a840fbc70 msgr2=0x7f9a840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- 172.21.15.96:0/1820601083 shutdown_connections 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- 172.21.15.96:0/1820601083 wait complete. 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 Processor -- start 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- start start 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 0x7f9a84197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a841022b0 0x7f9a8419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410c380 con 0x7f9a841018f0 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410c200 con 0x7f9a84100700 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a8c2de640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8410c500 con 0x7f9a841022b0 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a89879640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a89879640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40440/0 (socket says 172.21.15.96:40440) 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.313+0000 7f9a89879640 1 -- 172.21.15.96:0/3478073495 learned_addr learned my addr 172.21.15.96:0/3478073495 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a8a07a640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 0x7f9a84197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a8a87b640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a841022b0 0x7f9a8419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 -- 172.21.15.96:0/3478073495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a841022b0 msgr2=0x7f9a8419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a841022b0 0x7f9a8419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 -- 172.21.15.96:0/3478073495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 msgr2=0x7f9a84197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 0x7f9a84197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 -- 172.21.15.96:0/3478073495 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8419f730 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a8a07a640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 0x7f9a84197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a89879640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009230 tx=0x7f9a7402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74041070 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8419f9c0 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a74004450 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74003e20 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.314+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8419feb0 con 0x7f9a841018f0 2024-03-18T15:40:12.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.315+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a841a0080 con 0x7f9a841018f0 2024-03-18T15:40:12.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.316+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f9a74044070 con 0x7f9a841018f0 2024-03-18T15:40:12.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.317+0000 7f9a7b7fe640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 0x7f9a700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.317+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9a740f0a10 con 0x7f9a841018f0 2024-03-18T15:40:12.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.317+0000 7f9a8a07a640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 0x7f9a700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.317+0000 7f9a8a07a640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 0x7f9a700a4d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a84101750 tx=0x7f9a80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.320+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f9a740bd3e0 con 0x7f9a841018f0 2024-03-18T15:40:12.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.498+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a841a0410 con 0x7f9a841018f0 2024-03-18T15:40:12.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.499+0000 7f9a7b7fe640 1 -- 172.21.15.96:0/3478073495 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22715 (secure 0 0 0) 0x7f9a740c1480 con 0x7f9a841018f0 2024-03-18T15:40:12.500 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:12.500 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":90,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:40:02.631764+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:40:12.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 msgr2=0x7f9a700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 0x7f9a700a4d50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9a84101750 tx=0x7f9a80005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 msgr2=0x7f9a84198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009230 tx=0x7f9a7402c650 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 shutdown_connections 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a700a2890 0x7f9a700a4d50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a841022b0 0x7f9a8419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a841018f0 0x7f9a84198050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 --2- 172.21.15.96:0/3478073495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a84100700 0x7f9a84197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 >> 172.21.15.96:0/3478073495 conn(0x7f9a840fbc70 msgr2=0x7f9a840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 shutdown_connections 2024-03-18T15:40:12.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.501+0000 7f9a8c2de640 1 -- 172.21.15.96:0/3478073495 wait complete. 2024-03-18T15:40:12.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:40:12.514 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:40:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 -- 172.21.15.96:0/2576275113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2234103450 msgr2=0x7f2234109ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 --2- 172.21.15.96:0/2576275113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2234103450 0x7f2234109ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f222c009cd0 tx=0x7f222c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 -- 172.21.15.96:0/2576275113 shutdown_connections 2024-03-18T15:40:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 --2- 172.21.15.96:0/2576275113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2234103450 0x7f2234109ce0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 --2- 172.21.15.96:0/2576275113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234102a90 0x7f2234102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 --2- 172.21.15.96:0/2576275113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22340ffac0 0x7f22340ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 -- 172.21.15.96:0/2576275113 >> 172.21.15.96:0/2576275113 conn(0x7f22340fb3f0 msgr2=0x7f22340fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 -- 172.21.15.96:0/2576275113 shutdown_connections 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.617+0000 7f223aaf1640 1 -- 172.21.15.96:0/2576275113 wait complete. 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 Processor -- start 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 -- start start 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102a90 0x7f2234197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234103450 0x7f223419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f223410d470 con 0x7f22340ffac0 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f223410d2f0 con 0x7f2234103450 2024-03-18T15:40:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223aaf1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f223410d5f0 con 0x7f2234102a90 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40450/0 (socket says 172.21.15.96:40450) 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 -- 172.21.15.96:0/3290317285 learned_addr learned my addr 172.21.15.96:0/3290317285 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f2233fff640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102a90 0x7f2234197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223908e640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234103450 0x7f223419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 -- 172.21.15.96:0/3290317285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102a90 msgr2=0x7f2234197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102a90 0x7f2234197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 -- 172.21.15.96:0/3290317285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234103450 msgr2=0x7f223419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234103450 0x7f223419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 -- 172.21.15.96:0/3290317285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223419f6d0 con 0x7f22340ffac0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.618+0000 7f223888d640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f222800e9c0 tx=0x7f222800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.619+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222800cd80 con 0x7f22340ffac0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.619+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223419f9c0 con 0x7f22340ffac0 2024-03-18T15:40:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.619+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22280035f0 con 0x7f22340ffac0 2024-03-18T15:40:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.619+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228010690 con 0x7f22340ffac0 2024-03-18T15:40:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.619+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223419ff10 con 0x7f22340ffac0 2024-03-18T15:40:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.620+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2234068a20 con 0x7f22340ffac0 2024-03-18T15:40:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.621+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f22280037a0 con 0x7f22340ffac0 2024-03-18T15:40:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.621+0000 7f2231ffb640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 0x7f22240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.621+0000 7f2233fff640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 0x7f22240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.621+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f22280cefd0 con 0x7f22340ffac0 2024-03-18T15:40:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.622+0000 7f2233fff640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 0x7f22240a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f22200095f0 tx=0x7f2220005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.626+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f222809bee0 con 0x7f22340ffac0 2024-03-18T15:40:12.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.806+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f223402d230 con 0x7f22340ffac0 2024-03-18T15:40:12.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.806+0000 7f2231ffb640 1 -- 172.21.15.96:0/3290317285 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f222809b7e0 con 0x7f22340ffac0 2024-03-18T15:40:12.807 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:40:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 msgr2=0x7f22240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 0x7f22240a4d50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f22200095f0 tx=0x7f2220005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 msgr2=0x7f2234197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f222800e9c0 tx=0x7f222800ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 shutdown_connections 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22240a2890 0x7f22240a4d50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2234103450 0x7f223419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102a90 0x7f2234197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 --2- 172.21.15.96:0/3290317285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22340ffac0 0x7f2234197ab0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 >> 172.21.15.96:0/3290317285 conn(0x7f22340fb3f0 msgr2=0x7f2234105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 shutdown_connections 2024-03-18T15:40:12.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.808+0000 7f223aaf1640 1 -- 172.21.15.96:0/3290317285 wait complete. 2024-03-18T15:40:12.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:12 smithi096 ceph-mon[31930]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:12.819 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:12 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3478073495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:12.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:40:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:12 smithi137 ceph-mon[35868]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:12 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3478073495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:12.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:12 smithi193 ceph-mon[36702]: pgmap v355: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-03-18T15:40:12.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:12 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3478073495' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 -- 172.21.15.96:0/2102107483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 msgr2=0x7fade8106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 --2- 172.21.15.96:0/2102107483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8106fb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009900 tx=0x7fadd802bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 -- 172.21.15.96:0/2102107483 shutdown_connections 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 --2- 172.21.15.96:0/2102107483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 0x7fade81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 --2- 172.21.15.96:0/2102107483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8106fb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 --2- 172.21.15.96:0/2102107483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 0x7fade80ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 -- 172.21.15.96:0/2102107483 >> 172.21.15.96:0/2102107483 conn(0x7fade80fafa0 msgr2=0x7fade80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:12.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 -- 172.21.15.96:0/2102107483 shutdown_connections 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.923+0000 7fadedce8640 1 -- 172.21.15.96:0/2102107483 wait complete. 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 Processor -- start 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 -- start start 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 0x7fade8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 0x7fade819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fade810d450 con 0x7fade80ffd40 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fade810d2d0 con 0x7fade80ff3e0 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fadedce8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fade810d5d0 con 0x7fade81074f0 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40474/0 (socket says 172.21.15.96:40474) 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 -- 172.21.15.96:0/3155930576 learned_addr learned my addr 172.21.15.96:0/3155930576 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade77fe640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 0x7fade8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade7fff640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 0x7fade819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 -- 172.21.15.96:0/3155930576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 msgr2=0x7fade819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 0x7fade819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 -- 172.21.15.96:0/3155930576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 msgr2=0x7fade8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 0x7fade8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.924+0000 7fade6ffd640 1 -- 172.21.15.96:0/3155930576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819f770 con 0x7fade80ffd40 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fade6ffd640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000c00 tx=0x7fadd802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8041070 con 0x7fade80ffd40 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819fa00 con 0x7fade80ffd40 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd8004450 con 0x7fade80ffd40 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8034440 con 0x7fade80ffd40 2024-03-18T15:40:12.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.925+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade819ff40 con 0x7fade80ffd40 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.926+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade8105dc0 con 0x7fade80ffd40 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.927+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fadd8044070 con 0x7fade80ffd40 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.927+0000 7fade4ff9640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 0x7fadd40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.928+0000 7fade77fe640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 0x7fadd40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.928+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fadd80f05d0 con 0x7fade80ffd40 2024-03-18T15:40:12.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.928+0000 7fade77fe640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 0x7fadd40a4d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009bd0 tx=0x7fadd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:12.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:12.932+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fadd80bd030 con 0x7fade80ffd40 2024-03-18T15:40:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.103+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fade802d380 con 0x7fade80ffd40 2024-03-18T15:40:13.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.103+0000 7fade4ff9640 1 -- 172.21.15.96:0/3155930576 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fadd80c10d0 con 0x7fade80ffd40 2024-03-18T15:40:13.104 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:40:13.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 msgr2=0x7fadd40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 0x7fadd40a4d50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009bd0 tx=0x7fadd0005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 msgr2=0x7fade8198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fadd8000c00 tx=0x7fadd802c640 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 shutdown_connections 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fadd40a2890 0x7fadd40a4d50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fade81074f0 0x7fade819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade80ffd40 0x7fade8198090 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 --2- 172.21.15.96:0/3155930576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fade80ff3e0 0x7fade8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 >> 172.21.15.96:0/3155930576 conn(0x7fade80fafa0 msgr2=0x7fade8102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 shutdown_connections 2024-03-18T15:40:13.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.105+0000 7fadedce8640 1 -- 172.21.15.96:0/3155930576 wait complete. 2024-03-18T15:40:13.117 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 -- 172.21.15.96:0/3790275304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa944101810 msgr2=0x7fa944101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/3790275304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa944101810 0x7fa944101c30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009900 tx=0x7fa92c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 -- 172.21.15.96:0/3790275304 shutdown_connections 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/3790275304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9441033c0 0x7fa944109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/3790275304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 0x7fa944102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/3790275304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa944101810 0x7fa944101c30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.221+0000 7fa94a2dc640 1 -- 172.21.15.96:0/3790275304 >> 172.21.15.96:0/3790275304 conn(0x7fa9440fcd80 msgr2=0x7fa9440ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- 172.21.15.96:0/3790275304 shutdown_connections 2024-03-18T15:40:13.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- 172.21.15.96:0/3790275304 wait complete. 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 Processor -- start 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- start start 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa944101810 0x7fa944197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 0x7fa944198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410d490 con 0x7fa9441033c0 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410d310 con 0x7fa944102a00 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa94a2dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa94410d610 con 0x7fa944101810 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa948879640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa948879640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40492/0 (socket says 172.21.15.96:40492) 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.222+0000 7fa948879640 1 -- 172.21.15.96:0/519594267 learned_addr learned my addr 172.21.15.96:0/519594267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa9437fe640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 0x7fa944198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa943fff640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa944101810 0x7fa944197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 -- 172.21.15.96:0/519594267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa944101810 msgr2=0x7fa944197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa944101810 0x7fa944197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 -- 172.21.15.96:0/519594267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 msgr2=0x7fa944198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 0x7fa944198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 -- 172.21.15.96:0/519594267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94419f720 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa948879640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00eb10 tx=0x7fa93c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c004b20 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa93c004cc0 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94419fa10 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c010760 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.223+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94419ff60 con 0x7fa9441033c0 2024-03-18T15:40:13.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.224+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa944108b30 con 0x7fa9441033c0 2024-03-18T15:40:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.225+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fa93c005020 con 0x7fa9441033c0 2024-03-18T15:40:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.225+0000 7fa9417fa640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.226+0000 7fa943fff640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.226+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa93c0cf300 con 0x7fa9441033c0 2024-03-18T15:40:13.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.226+0000 7fa943fff640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 0x7fa9300a4d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000c00 tx=0x7fa92c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.230+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fa93c09c210 con 0x7fa9441033c0 2024-03-18T15:40:13.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.408+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9440623e0 con 0x7fa9441033c0 2024-03-18T15:40:13.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.409+0000 7fa9417fa640 1 -- 172.21.15.96:0/519594267 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fa93c09bb10 con 0x7fa9441033c0 2024-03-18T15:40:13.410 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 msgr2=0x7fa9300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 0x7fa9300a4d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000c00 tx=0x7fa92c005fd0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 msgr2=0x7fa94419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00eb10 tx=0x7fa93c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 shutdown_connections 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa9300a2890 0x7fa9300a4d50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9441033c0 0x7fa94419f0b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa944102a00 0x7fa944198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 --2- 172.21.15.96:0/519594267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa944101810 0x7fa944197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 >> 172.21.15.96:0/519594267 conn(0x7fa9440fcd80 msgr2=0x7fa944105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 shutdown_connections 2024-03-18T15:40:13.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.411+0000 7fa94a2dc640 1 -- 172.21.15.96:0/519594267 wait complete. 2024-03-18T15:40:13.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.523+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/4009441800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 msgr2=0x7f5e74101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/4009441800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e74101d70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009900 tx=0x7f5e7002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/4009441800 shutdown_connections 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/4009441800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e741022b0 0x7f5e74108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/4009441800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e74101d70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/4009441800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e74100700 0x7f5e74100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/4009441800 >> 172.21.15.96:0/4009441800 conn(0x7f5e740fbc70 msgr2=0x7f5e740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/4009441800 shutdown_connections 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/4009441800 wait complete. 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 Processor -- start 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- start start 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e74100700 0x7f5e740694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 0x7f5e7419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7410c380 con 0x7f5e741018f0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7410c200 con 0x7f5e741022b0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e7bd5b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7410c500 con 0x7f5e74100700 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.524+0000 7f5e792f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40508/0 (socket says 172.21.15.96:40508) 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e79af7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e74100700 0x7f5e740694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 -- 172.21.15.96:0/1616139061 learned_addr learned my addr 172.21.15.96:0/1616139061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e7a2f8640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 0x7f5e7419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 -- 172.21.15.96:0/1616139061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e74100700 msgr2=0x7f5e740694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e74100700 0x7f5e740694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 -- 172.21.15.96:0/1616139061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 msgr2=0x7f5e7419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 0x7f5e7419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 -- 172.21.15.96:0/1616139061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7419f7a0 con 0x7f5e741018f0 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e7a2f8640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 0x7f5e7419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:13.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e792f6640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009230 tx=0x7f5e7002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70041070 con 0x7f5e741018f0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7419f9d0 con 0x7f5e741018f0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e70004450 con 0x7f5e741018f0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003e20 con 0x7f5e741018f0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.525+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7419ff10 con 0x7f5e741018f0 2024-03-18T15:40:13.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.526+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e74107a20 con 0x7f5e741018f0 2024-03-18T15:40:13.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.527+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f5e70044070 con 0x7f5e741018f0 2024-03-18T15:40:13.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.528+0000 7f5e6affd640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 0x7f5e600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.528+0000 7f5e79af7640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 0x7f5e600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.528+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5e700f0a20 con 0x7f5e741018f0 2024-03-18T15:40:13.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.528+0000 7f5e79af7640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 0x7f5e600a4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e640095f0 tx=0x7f5e64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.532+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5e700bd3f0 con 0x7f5e741018f0 2024-03-18T15:40:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:13 smithi096 ceph-mon[31930]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:13 smithi096 ceph-mon[31930]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:13 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3290317285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:13 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3155930576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:13 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/519594267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.746+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e7402d1b0 con 0x7f5e741018f0 2024-03-18T15:40:13.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.747+0000 7f5e6affd640 1 -- 172.21.15.96:0/1616139061 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7f5e700c1490 con 0x7f5e741018f0 2024-03-18T15:40:13.748 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:13.748 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:40:03.325244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:40:13.748 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 msgr2=0x7f5e600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 0x7f5e600a4d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5e640095f0 tx=0x7f5e64005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 msgr2=0x7f5e7419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009230 tx=0x7f5e7002c650 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 shutdown_connections 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5e600a2890 0x7f5e600a4d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e741022b0 0x7f5e7419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e741018f0 0x7f5e7419c870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 --2- 172.21.15.96:0/1616139061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e74100700 0x7f5e740694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 >> 172.21.15.96:0/1616139061 conn(0x7f5e740fbc70 msgr2=0x7f5e74104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 shutdown_connections 2024-03-18T15:40:13.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.749+0000 7f5e7bd5b640 1 -- 172.21.15.96:0/1616139061 wait complete. 2024-03-18T15:40:13.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:40:13.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:13 smithi137 ceph-mon[35868]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:13 smithi137 ceph-mon[35868]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:13 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3290317285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:13 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3155930576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:13 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/519594267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.861+0000 7fe489812640 1 -- 172.21.15.96:0/762825074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4841022b0 msgr2=0x7fe484108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.861+0000 7fe489812640 1 --2- 172.21.15.96:0/762825074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4841022b0 0x7fe484108b40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009ca0 tx=0x7fe47c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- 172.21.15.96:0/762825074 shutdown_connections 2024-03-18T15:40:13.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- 172.21.15.96:0/762825074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4841022b0 0x7fe484108b40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- 172.21.15.96:0/762825074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 0x7fe484101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- 172.21.15.96:0/762825074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe484100700 0x7fe484100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- 172.21.15.96:0/762825074 >> 172.21.15.96:0/762825074 conn(0x7fe4840fbc90 msgr2=0x7fe4840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- 172.21.15.96:0/762825074 shutdown_connections 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- 172.21.15.96:0/762825074 wait complete. 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 Processor -- start 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- start start 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 0x7fe48419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4841022b0 0x7fe48419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410c380 con 0x7fe484100700 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410c200 con 0x7fe4841022b0 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.862+0000 7fe489812640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410c500 con 0x7fe4841018f0 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57736/0 (socket says 172.21.15.96:57736) 2024-03-18T15:40:13.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 -- 172.21.15.96:0/511485691 learned_addr learned my addr 172.21.15.96:0/511485691 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe4827fc640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 0x7fe48419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe4837fe640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4841022b0 0x7fe48419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 -- 172.21.15.96:0/511485691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 msgr2=0x7fe48419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 0x7fe48419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 -- 172.21.15.96:0/511485691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4841022b0 msgr2=0x7fe48419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4841022b0 0x7fe48419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 -- 172.21.15.96:0/511485691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48419f6c0 con 0x7fe484100700 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe482ffd640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe47800e9c0 tx=0x7fe47800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800cd80 con 0x7fe484100700 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4780035f0 con 0x7fe484100700 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478010690 con 0x7fe484100700 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48419f9b0 con 0x7fe484100700 2024-03-18T15:40:13.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.863+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48419ff50 con 0x7fe484100700 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.864+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4841079c0 con 0x7fe484100700 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.865+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fe4780037a0 con 0x7fe484100700 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.866+0000 7fe488810640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.866+0000 7fe4827fc640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.866+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe4780cef30 con 0x7fe484100700 2024-03-18T15:40:13.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.866+0000 7fe4827fc640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe48406a6b0 tx=0x7fe474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:13.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:13.870+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fe47809be40 con 0x7fe484100700 2024-03-18T15:40:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:13 smithi193 ceph-mon[36702]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:13 smithi193 ceph-mon[36702]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-18T15:40:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:13 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3290317285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:13 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3155930576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:40:13.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:13 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/519594267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:40:14.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.047+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe48402d330 con 0x7fe484100700 2024-03-18T15:40:14.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.048+0000 7fe488810640 1 -- 172.21.15.96:0/511485691 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22715 (secure 0 0 0) 0x7fe47809b740 con 0x7fe484100700 2024-03-18T15:40:14.049 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:14.049 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":90,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:40:02.631764+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 msgr2=0x7fe46c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe48406a6b0 tx=0x7fe474005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 msgr2=0x7fe48406a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe47800e9c0 tx=0x7fe47800ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 shutdown_connections 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe46c0a2910 0x7fe46c0a4dd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4841022b0 0x7fe48419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4841018f0 0x7fe48419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 --2- 172.21.15.96:0/511485691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe484100700 0x7fe48406a940 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 >> 172.21.15.96:0/511485691 conn(0x7fe4840fbc90 msgr2=0x7fe484104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 shutdown_connections 2024-03-18T15:40:14.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.050+0000 7fe489812640 1 -- 172.21.15.96:0/511485691 wait complete. 2024-03-18T15:40:14.063 DEBUG:tasks.fwd_scrub:Yielding 2024-03-18T15:40:14.063 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-18T15:40:14.073 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:40:14.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-18T15:40:14.074 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-18T15:40:14.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:40:14.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:14.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:40:14.076 INFO:teuthology.run_tasks:Running task workunit... 2024-03-18T15:40:14.088 INFO:tasks.workunit:Pulling workunits from ref 4e19f704afd9957ea657ab858f9ecf76401fef6f 2024-03-18T15:40:14.089 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-18T15:40:14.089 INFO:tasks.workunit:timeout=3h 2024-03-18T15:40:14.089 INFO:tasks.workunit:cleanup=True 2024-03-18T15:40:14.089 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-18T15:40:14.104 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-03-18 15:38:10.548989050 +0000 2024-03-18T15:40:14.105 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-03-18 15:38:10.548989050 +0000 2024-03-18T15:40:14.105 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-03-18 15:40:02.609210220 +0000 2024-03-18T15:40:14.105 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-03-18 15:38:10.548989050 +0000 2024-03-18T15:40:14.105 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-18T15:40:14.105 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-18T15:40:14.180 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4e19f704afd9957ea657ab858f9ecf76401fef6f 2024-03-18T15:40:14.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 -- 172.21.15.96:0/16499053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601029e0 msgr2=0x7fb160102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 --2- 172.21.15.96:0/16499053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601029e0 0x7fb160102e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb148009900 tx=0x7fb14802bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:14.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 -- 172.21.15.96:0/16499053 shutdown_connections 2024-03-18T15:40:14.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 --2- 172.21.15.96:0/16499053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 0x7fb160109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 --2- 172.21.15.96:0/16499053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601029e0 0x7fb160102e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 --2- 172.21.15.96:0/16499053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601017f0 0x7fb160101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.181+0000 7fb1652b2640 1 -- 172.21.15.96:0/16499053 >> 172.21.15.96:0/16499053 conn(0x7fb1600fcd40 msgr2=0x7fb1600ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- 172.21.15.96:0/16499053 shutdown_connections 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- 172.21.15.96:0/16499053 wait complete. 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 Processor -- start 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- start start 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601029e0 0x7fb1601980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 0x7fb16019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb16010d450 con 0x7fb1601017f0 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb16010d2d0 con 0x7fb1601033a0 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb1652b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb16010d5d0 con 0x7fb1601029e0 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb15effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb15effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57764/0 (socket says 172.21.15.96:57764) 2024-03-18T15:40:14.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb15effd640 1 -- 172.21.15.96:0/2063201105 learned_addr learned my addr 172.21.15.96:0/2063201105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb157fff640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601029e0 0x7fb1601980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.182+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 0x7fb16019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 -- 172.21.15.96:0/2063201105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601029e0 msgr2=0x7fb1601980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601029e0 0x7fb1601980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 -- 172.21.15.96:0/2063201105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 msgr2=0x7fb16019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 0x7fb16019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 -- 172.21.15.96:0/2063201105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16019f780 con 0x7fb1601017f0 2024-03-18T15:40:14.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15effd640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb15000e9c0 tx=0x7fb15000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15cff9640 1 -- 172.21.15.96:0/2063201105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15000ccc0 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16019fa70 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15cff9640 1 -- 172.21.15.96:0/2063201105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1500035f0 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb15cff9640 1 -- 172.21.15.96:0/2063201105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15000ce60 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.183+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16019ff90 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.185+0000 7fb15cff9640 1 -- 172.21.15.96:0/2063201105 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fb1500037a0 con 0x7fb1601017f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 Processor -- start 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 -- start start 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 0x563a53f3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f3b4f0 0x563a53f3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15cff9640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 0x7fb14c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a53e9a0d0 con 0x563a53f35990 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a53e42c70 con 0x563a53f3b4f0 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb167f80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a53e96170 con 0x563a53f36380 2024-03-18T15:40:14.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 0x563a53f3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb157fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f3b4f0 0x563a53f3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57768/0 (socket says 172.21.15.96:57768) 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15f7fe640 1 -- 172.21.15.96:0/554477850 learned_addr learned my addr 172.21.15.96:0/554477850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 0x7fb14c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.186+0000 7fb15cff9640 1 -- 172.21.15.96:0/2063201105 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb1500ceff0 con 0x7fb1601017f0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 -- 172.21.15.96:0/554477850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f3b4f0 msgr2=0x563a53f3d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f3b4f0 0x563a53f3d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 -- 172.21.15.96:0/554477850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 msgr2=0x563a53f35db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 -- 172.21.15.96:0/554477850 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a53c767e0 con 0x563a53f36380 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 0x7fb14c0a4d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x563a53f36be0 tx=0x7fb158009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb15effd640 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 0x563a53f3ae80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1500cc970 tx=0x7fb15006e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd190eeabbe40436 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb1557fa640 1 -- 172.21.15.96:0/554477850 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15009bf00 con 0x563a53f36380 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb1557fa640 1 -- 172.21.15.96:0/554477850 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb150066a20 con 0x563a53f36380 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.187+0000 7fb1557fa640 1 -- 172.21.15.96:0/554477850 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1500a3fa0 con 0x563a53f36380 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- 172.21.15.96:0/554477850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 msgr2=0x563a53f3ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 0x563a53f3ae80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1500cc970 tx=0x7fb15006e530 comp rx=0 tx=0).stop 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- 172.21.15.96:0/554477850 shutdown_connections 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f3b4f0 0x563a53f3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f36380 0x563a53f3ae80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 --2- 172.21.15.96:0/554477850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f35990 0x563a53f35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- 172.21.15.96:0/554477850 >> 172.21.15.96:0/554477850 conn(0x563a53efdc50 msgr2=0x563a53f31300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- 172.21.15.96:0/554477850 shutdown_connections 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- 172.21.15.96:0/554477850 wait complete. 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 -- start start 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.188+0000 7fb167f80740 1 Processor -- start 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f47370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563a53f3e3e0 con 0x563a53f3b4f0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563a53e560d0 con 0x563a53f36380 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a53f3e560 con 0x563a53f35990 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 10 client.0 _ll_register_callbacks cb 0x563a53e1f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f47370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:55346/0 (socket says 172.21.15.96:55346) 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15effd640 1 -- 172.21.15.96:0/4269301999 learned_addr learned my addr 172.21.15.96:0/4269301999 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb157fff640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 -- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 msgr2=0x563a53f3fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 -- 172.21.15.96:0/4269301999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 msgr2=0x563a53f40330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f40330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 -- 172.21.15.96:0/4269301999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a53cd6f70 con 0x563a53f3b4f0 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb157fff640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f40330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15effd640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f47370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb158005fd0 tx=0x7fb158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c4519bd8c1306 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb1467fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158004d20 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb158004480 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.189+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb150066a20 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb158004680 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a53b6e170 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb150066a20 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb158027030 con 0x563a53f3b4f0 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 20 client.15510 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53f4b910 0x563a53f4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 4 client.15510 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a53ea7610 con 0x563a53f4b910 2024-03-18T15:40:14.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb15effd640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53f4b910 0x563a53f4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb1467fc640 10 client.15510 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.190+0000 7fb15effd640 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53f4b910 0x563a53f4dd80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c555dd8dfb38a71 server_cookie=1f218c9f2aa8a631 in_seq=0 out_seq=0 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.191+0000 7fb1467fc640 1 -- 172.21.15.96:0/4269301999 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563a53ea7610 con 0x563a53f4b910 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.191+0000 7fb1467fc640 10 client.15510 handle_command_reply: tid=0 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.191+0000 7fb167f80740 1 client.15510 shutdown 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 msgr2=0x563a53f47370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f47370 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb158005fd0 tx=0x7fb158004a40 comp rx=0 tx=0).stop 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 shutdown_connections 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53f4b910 0x563a53f4dd80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb1467fc640 0 client.15510 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f47370 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f40330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 --2- 172.21.15.96:0/4269301999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 >> 172.21.15.96:0/4269301999 conn(0x563a53efdc50 msgr2=0x563a53f304e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 shutdown_connections 2024-03-18T15:40:14.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 1 -- 172.21.15.96:0/4269301999 wait complete. 2024-03-18T15:40:14.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.192+0000 7fb167f80740 20 client.15510 trim_cache size 0 max 16384 2024-03-18T15:40:14.199 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-18T15:40:14.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.209+0000 7fb167f80740 1 Processor -- start 2024-03-18T15:40:14.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.209+0000 7fb167f80740 1 -- start start 2024-03-18T15:40:14.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601a8ea0 con 0x563a53f3b4f0 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601aa9e0 con 0x563a53f36380 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb167f80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601a9570 con 0x563a53f35990 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb157fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:55360/0 (socket says 172.21.15.96:55360) 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57790/0 (socket says 172.21.15.96:57790) 2024-03-18T15:40:14.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15effd640 1 -- 172.21.15.96:0/3081704660 learned_addr learned my addr 172.21.15.96:0/3081704660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 -- 172.21.15.96:0/3081704660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 msgr2=0x563a53f34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 -- 172.21.15.96:0/3081704660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 msgr2=0x563a53f47330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f47330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 -- 172.21.15.96:0/3081704660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a53f318a0 con 0x563a53f3b4f0 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb157fff640 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15effd640 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.210+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1580191a0 tx=0x7fb1580049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb079644d23c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb154ff9640 1 -- 172.21.15.96:0/3081704660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158037070 con 0x563a53f3b4f0 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb154ff9640 1 -- 172.21.15.96:0/3081704660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb158020e30 con 0x563a53f3b4f0 2024-03-18T15:40:14.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb154ff9640 1 -- 172.21.15.96:0/3081704660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580337c0 con 0x563a53f3b4f0 2024-03-18T15:40:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 -- 172.21.15.96:0/3081704660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 msgr2=0x563a53f52640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1580191a0 tx=0x7fb1580049b0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 -- 172.21.15.96:0/3081704660 shutdown_connections 2024-03-18T15:40:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f3b4f0 0x563a53f52640 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f36380 0x563a53f47330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 --2- 172.21.15.96:0/3081704660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f35990 0x563a53f34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 -- 172.21.15.96:0/3081704660 >> 172.21.15.96:0/3081704660 conn(0x563a53efdc50 msgr2=0x563a53f34540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.211+0000 7fb167f80740 1 -- 172.21.15.96:0/3081704660 shutdown_connections 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 -- 172.21.15.96:0/3081704660 wait complete. 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 -- start start 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 Processor -- start 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f35990 0x563a53f45220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f3b4f0 0x563a53f46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601aa310 con 0x563a53f36380 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601a9c40 con 0x563a53f35990 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb1601a6e80 con 0x563a53f3b4f0 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb167f80740 10 client.0 _ll_register_callbacks cb 0x7fb158011620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57800/0 (socket says 172.21.15.96:57800) 2024-03-18T15:40:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 -- 172.21.15.96:0/563541291 learned_addr learned my addr 172.21.15.96:0/563541291 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb15effd640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f35990 0x563a53f45220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb15f7fe640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f3b4f0 0x563a53f46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 -- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f3b4f0 msgr2=0x563a53f46770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f3b4f0 0x563a53f46770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 -- 172.21.15.96:0/563541291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f35990 msgr2=0x563a53f45220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f35990 0x563a53f45220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.212+0000 7fb157fff640 1 -- 172.21.15.96:0/563541291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a53f46d60 con 0x563a53f36380 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb157fff640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb148036040 tx=0x7fb148004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82da9a37301d0aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14804b060 con 0x563a53f36380 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb148005450 con 0x563a53f36380 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb158020e30 con 0x563a53f36380 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb148005650 con 0x563a53f36380 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 10 client.15516 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a53fb2560 con 0x563a53f36380 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb158020e30 con 0x563a53f36380 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb148003ba0 con 0x563a53f36380 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 10 client.15516 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 20 client.15516 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53fb2f90 0x563a53fb53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 4 client.15516 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a53ea1960 con 0x563a53fb2f90 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.213+0000 7fb15effd640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53fb2f90 0x563a53fb53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.214+0000 7fb1467fc640 10 client.15516 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.214+0000 7fb15effd640 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53fb2f90 0x563a53fb53b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff778b7c84f32151 server_cookie=c5c5b36855f76f6a in_seq=0 out_seq=0 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb1467fc640 1 -- 172.21.15.96:0/563541291 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a53ea1960 con 0x563a53fb2f90 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb1467fc640 10 client.15516 handle_command_reply: tid=0 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 client.15516 shutdown 2024-03-18T15:40:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 msgr2=0x563a53f45760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb148036040 tx=0x7fb148004610 comp rx=0 tx=0).stop 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 shutdown_connections 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53f3b4f0 0x563a53f46770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563a53fb2f90 0x563a53fb53b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563a53f36380 0x563a53f45760 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 --2- 172.21.15.96:0/563541291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563a53f35990 0x563a53f45220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 >> 172.21.15.96:0/563541291 conn(0x563a53efdc50 msgr2=0x563a53f34dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb1467fc640 0 client.15516 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 shutdown_connections 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 1 -- 172.21.15.96:0/563541291 wait complete. 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.215+0000 7fb167f80740 20 client.15516 trim_cache size 0 max 16384 2024-03-18T15:40:14.222 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:40:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:40:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "aef49789-18ca-4fb6-8a6e-1a5052e897b7", 2024-03-18T15:40:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:40:14.223 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 msgr2=0x7fb14c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 0x7fb14c0a4d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x563a53f36be0 tx=0x7fb158009500 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 msgr2=0x7fb160197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb15000e9c0 tx=0x7fb15000edc0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 shutdown_connections 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb14c0a2890 0x7fb14c0a4d50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb1601033a0 0x7fb16019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1601029e0 0x7fb1601980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 --2- 172.21.15.96:0/2063201105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1601017f0 0x7fb160197bb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 >> 172.21.15.96:0/2063201105 conn(0x7fb1600fcd40 msgr2=0x7fb160105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 shutdown_connections 2024-03-18T15:40:14.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.216+0000 7fb1652b2640 1 -- 172.21.15.96:0/2063201105 wait complete. 2024-03-18T15:40:14.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:40:14.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 -- 172.21.15.96:0/2647337160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41029e0 msgr2=0x7fefa4102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 --2- 172.21.15.96:0/2647337160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41029e0 0x7fefa4102e60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef94009900 tx=0x7fef9402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 -- 172.21.15.96:0/2647337160 shutdown_connections 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 --2- 172.21.15.96:0/2647337160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 0x7fefa4109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 --2- 172.21.15.96:0/2647337160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41029e0 0x7fefa4102e60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 --2- 172.21.15.96:0/2647337160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41017f0 0x7fefa4101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 -- 172.21.15.96:0/2647337160 >> 172.21.15.96:0/2647337160 conn(0x7fefa40fcd80 msgr2=0x7fefa40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 -- 172.21.15.96:0/2647337160 shutdown_connections 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.371+0000 7fefac5f4640 1 -- 172.21.15.96:0/2647337160 wait complete. 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 Processor -- start 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 -- start start 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41029e0 0x7fefa4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 0x7fefa419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefa410d470 con 0x7fefa41017f0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa410d2f0 con 0x7fefa41033a0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefac5f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefa410d5f0 con 0x7fefa41029e0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefaa390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefaa390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57832/0 (socket says 172.21.15.96:57832) 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefaa390640 1 -- 172.21.15.96:0/1494308579 learned_addr learned my addr 172.21.15.96:0/1494308579 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefaab91640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 0x7fefa419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.372+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41029e0 0x7fefa4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 -- 172.21.15.96:0/1494308579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41029e0 msgr2=0x7fefa4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41029e0 0x7fefa4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 -- 172.21.15.96:0/1494308579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 msgr2=0x7fefa419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 0x7fefa419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 -- 172.21.15.96:0/1494308579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa419f740 con 0x7fefa41017f0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefaa390640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e9c0 tx=0x7fef9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fef9b7fe640 1 -- 172.21.15.96:0/1494308579 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00cd80 con 0x7fefa41017f0 2024-03-18T15:40:14.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fef9b7fe640 1 -- 172.21.15.96:0/1494308579 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c0035f0 con 0x7fefa41017f0 2024-03-18T15:40:14.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fef9b7fe640 1 -- 172.21.15.96:0/1494308579 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c010690 con 0x7fefa41017f0 2024-03-18T15:40:14.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa419fa30 con 0x7fefa41017f0 2024-03-18T15:40:14.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.373+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa419ff50 con 0x7fefa41017f0 2024-03-18T15:40:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fef9b7fe640 1 -- 172.21.15.96:0/1494308579 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fef9c0037a0 con 0x7fefa41017f0 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 Processor -- start 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 -- start start 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5627a1312500 con 0x5627a13adb60 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5627a12babe0 con 0x5627a13b2cd0 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5627a130e5a0 con 0x5627a13ad170 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.376+0000 7fef9b7fe640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 0x7fef900a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57848/0 (socket says 172.21.15.96:57848) 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaab91640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 -- 172.21.15.96:0/1995822332 learned_addr learned my addr 172.21.15.96:0/1995822332 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaab91640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 0x7fef900a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fef9b7fe640 1 -- 172.21.15.96:0/1494308579 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fef9c0ceef0 con 0x7fefa41017f0 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 -- 172.21.15.96:0/1995822332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 msgr2=0x5627a13ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 -- 172.21.15.96:0/1995822332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 msgr2=0x5627a13b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 -- 172.21.15.96:0/1995822332 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a12ce4b0 con 0x5627a13adb60 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaab91640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaa390640 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fef9c004630 tx=0x7fef9c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c02bd149f8cd5de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fefaab91640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 0x7fef900a4dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5627a13ae3c0 tx=0x7fefa0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fef98bfe640 1 -- 172.21.15.96:0/1995822332 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c09b700 con 0x5627a13adb60 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fef98bfe640 1 -- 172.21.15.96:0/1995822332 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c09f850 con 0x5627a13adb60 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.377+0000 7fef98bfe640 1 -- 172.21.15.96:0/1995822332 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0a3a10 con 0x5627a13adb60 2024-03-18T15:40:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- 172.21.15.96:0/1995822332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 msgr2=0x5627a13b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fef9c004630 tx=0x7fef9c0097b0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- 172.21.15.96:0/1995822332 shutdown_connections 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b2660 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/1995822332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- 172.21.15.96:0/1995822332 >> 172.21.15.96:0/1995822332 conn(0x5627a13a86e0 msgr2=0x5627a13a8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- 172.21.15.96:0/1995822332 shutdown_connections 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- 172.21.15.96:0/1995822332 wait complete. 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 -- start start 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.378+0000 7fefaf1e4740 1 Processor -- start 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5627a13b5bc0 con 0x5627a13adb60 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5627a12ce160 con 0x5627a13b2cd0 2024-03-18T15:40:14.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5627a13b5d40 con 0x5627a13ad170 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 10 client.0 _ll_register_callbacks cb 0x5627a12b2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57860/0 (socket says 172.21.15.96:57860) 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 -- 172.21.15.96:0/4039036088 learned_addr learned my addr 172.21.15.96:0/4039036088 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaa390640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaab91640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 -- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 msgr2=0x5627a13b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13b7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 -- 172.21.15.96:0/4039036088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 msgr2=0x5627a13bebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13bebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 -- 172.21.15.96:0/4039036088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a10f07d0 con 0x5627a13adb60 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef94036040 tx=0x7fef94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effca20ca5f7056a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94005d50 con 0x5627a13adb60 2024-03-18T15:40:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fef94005230 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.379+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c09f850 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef9404b030 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627a10263f0 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef9c09f850 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fef94003710 con 0x5627a13adb60 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 20 client.15528 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a13c31d0 0x5627a13c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 4 client.15528 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627a131fa40 con 0x5627a13c31d0 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaa390640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a13c31d0 0x5627a13c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fef8dffb640 10 client.15528 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.380+0000 7fefaa390640 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a13c31d0 0x5627a13c5600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5130263c1b7d1b38 server_cookie=b17d65a9f9482464 in_seq=0 out_seq=0 2024-03-18T15:40:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.381+0000 7fef8dffb640 1 -- 172.21.15.96:0/4039036088 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5627a131fa40 con 0x5627a13c31d0 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.381+0000 7fef8dffb640 10 client.15528 handle_command_reply: tid=0 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.381+0000 7fefaf1e4740 1 client.15528 shutdown 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.381+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 msgr2=0x5627a13b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.381+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fef94036040 tx=0x7fef94004270 comp rx=0 tx=0).stop 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 shutdown_connections 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13b2cd0 0x5627a13bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a13c31d0 0x5627a13c5600 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13b7ba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4039036088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fef8dffb640 0 client.15528 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 >> 172.21.15.96:0/4039036088 conn(0x5627a13a86e0 msgr2=0x5627a13a7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 shutdown_connections 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4039036088 wait complete. 2024-03-18T15:40:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.382+0000 7fefaf1e4740 20 client.15528 trim_cache size 0 max 16384 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 Processor -- start 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 -- start start 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 0x5627a13af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13b2cd0 0x5627a13beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a8db0 con 0x5627a13adb60 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41aa8f0 con 0x5627a13ad170 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a9480 con 0x5627a13b2cd0 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaa390640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 0x5627a13af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57872/0 (socket says 172.21.15.96:57872) 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaa390640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 0x5627a13af0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:59508/0 (socket says 172.21.15.96:59508) 2024-03-18T15:40:14.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 -- 172.21.15.96:0/2468913236 learned_addr learned my addr 172.21.15.96:0/2468913236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefaab91640 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13b2cd0 0x5627a13beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 -- 172.21.15.96:0/2468913236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13b2cd0 msgr2=0x5627a13beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13b2cd0 0x5627a13beba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 -- 172.21.15.96:0/2468913236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 msgr2=0x5627a13af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 0x5627a13af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 -- 172.21.15.96:0/2468913236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a0fe60f0 con 0x5627a13adb60 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.399+0000 7fefa9b8f640 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fef9402c170 tx=0x7fef9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185a0f8c1ffdbebf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fef8ffff640 1 -- 172.21.15.96:0/2468913236 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94005d50 con 0x5627a13adb60 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fef8ffff640 1 -- 172.21.15.96:0/2468913236 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef94041070 con 0x5627a13adb60 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fef8ffff640 1 -- 172.21.15.96:0/2468913236 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef94003850 con 0x5627a13adb60 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 -- 172.21.15.96:0/2468913236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 msgr2=0x5627a13af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fef9402c170 tx=0x7fef9402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 -- 172.21.15.96:0/2468913236 shutdown_connections 2024-03-18T15:40:14.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13b2cd0 0x5627a13beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13adb60 0x5627a13af630 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/2468913236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13ad170 0x5627a13af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 -- 172.21.15.96:0/2468913236 >> 172.21.15.96:0/2468913236 conn(0x5627a13a86e0 msgr2=0x5627a13a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 -- 172.21.15.96:0/2468913236 shutdown_connections 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.400+0000 7fefaf1e4740 1 -- 172.21.15.96:0/2468913236 wait complete. 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 -- start start 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 Processor -- start 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13adb60 0x5627a13bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41aa220 con 0x5627a13b2cd0 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a9b50 con 0x5627a13adb60 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefa41a6d90 con 0x5627a13ad170 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaf1e4740 10 client.0 _ll_register_callbacks cb 0x7fef94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:14.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefa9b8f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13adb60 0x5627a13bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57884/0 (socket says 172.21.15.96:57884) 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaa390640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 -- 172.21.15.96:0/4016062550 learned_addr learned my addr 172.21.15.96:0/4016062550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 -- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 msgr2=0x5627a13bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13bc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 -- 172.21.15.96:0/4016062550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13adb60 msgr2=0x5627a13bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13adb60 0x5627a13bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 -- 172.21.15.96:0/4016062550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a12d99d0 con 0x5627a13b2cd0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fefaab91640 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefa0003bb0 tx=0x7fefa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b658274051942260 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.401+0000 7fef8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0030060 con 0x5627a13b2cd0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fefa0020450 con 0x5627a13b2cd0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef94041070 con 0x5627a13b2cd0 2024-03-18T15:40:14.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefa0020650 con 0x5627a13b2cd0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 10 client.15534 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5627a13bef10 con 0x5627a13b2cd0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef94041070 con 0x5627a13b2cd0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fefa002a070 con 0x5627a13b2cd0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 10 client.15534 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 20 client.15534 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a1429ca0 0x5627a142c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 4 client.15534 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627a1319d90 con 0x5627a1429ca0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.402+0000 7fefaa390640 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a1429ca0 0x5627a142c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fef8dffb640 10 client.15534 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaa390640 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a1429ca0 0x5627a142c0c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c248838d449461 server_cookie=923a92d2be12cb88 in_seq=0 out_seq=0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fef8dffb640 1 -- 172.21.15.96:0/4016062550 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5627a1319d90 con 0x5627a1429ca0 2024-03-18T15:40:14.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fef8dffb640 10 client.15534 handle_command_reply: tid=0 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 client.15534 shutdown 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 msgr2=0x5627a13bd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fefa0003bb0 tx=0x7fefa0004a40 comp rx=0 tx=0).stop 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 shutdown_connections 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5627a1429ca0 0x5627a142c0c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5627a13b2cd0 0x5627a13bd770 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627a13adb60 0x5627a13bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fef8dffb640 0 client.15534 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 --2- 172.21.15.96:0/4016062550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5627a13ad170 0x5627a13bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 >> 172.21.15.96:0/4016062550 conn(0x5627a13a86e0 msgr2=0x5627a13b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 shutdown_connections 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 1 -- 172.21.15.96:0/4016062550 wait complete. 2024-03-18T15:40:14.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefaf1e4740 20 client.15534 trim_cache size 0 max 16384 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "aef49789-18ca-4fb6-8a6e-1a5052e897b7": { 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "aef49789-18ca-4fb6-8a6e-1a5052e897b7", 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:40:14.411 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 msgr2=0x7fef900a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.403+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 0x7fef900a4dd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x5627a13ae3c0 tx=0x7fefa0009500 comp rx=0 tx=0).stop 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 msgr2=0x7fefa4197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e9c0 tx=0x7fef9c00ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 shutdown_connections 2024-03-18T15:40:14.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef900a2910 0x7fef900a4dd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefa41033a0 0x7fefa419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefa41029e0 0x7fefa4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 --2- 172.21.15.96:0/1494308579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa41017f0 0x7fefa4197b20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 >> 172.21.15.96:0/1494308579 conn(0x7fefa40fcd80 msgr2=0x7fefa4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 shutdown_connections 2024-03-18T15:40:14.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:14.404+0000 7fefac5f4640 1 -- 172.21.15.96:0/1494308579 wait complete. 2024-03-18T15:40:14.415 INFO:tasks.cephfs.filesystem:scrub status for tag:aef49789-18ca-4fb6-8a6e-1a5052e897b7 - {'path': '/', 'tag': 'aef49789-18ca-4fb6-8a6e-1a5052e897b7', 'options': 'recursive,force'} 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1616139061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/511485691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:40:14.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:14 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:40:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1616139061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/511485691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:14 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: pgmap v356: 97 pgs: 97 active+clean; 652 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1616139061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/511485691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:14 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:40:16.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:16 smithi137 ceph-mon[35868]: pgmap v357: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:16 smithi193 ceph-mon[36702]: pgmap v357: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:16.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:16 smithi096 ceph-mon[31930]: pgmap v357: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 12 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:18.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:18 smithi137 ceph-mon[35868]: pgmap v358: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:18 smithi193 ceph-mon[36702]: pgmap v358: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:18 smithi096 ceph-mon[31930]: pgmap v358: 97 pgs: 97 active+clean; 657 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-03-18T15:40:20.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:20 smithi137 ceph-mon[35868]: pgmap v359: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:20 smithi193 ceph-mon[36702]: pgmap v359: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:20 smithi096 ceph-mon[31930]: pgmap v359: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:22.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:22 smithi137 ceph-mon[35868]: pgmap v360: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:22 smithi193 ceph-mon[36702]: pgmap v360: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:22.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:22 smithi096 ceph-mon[31930]: pgmap v360: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:23.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:23 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:40:23.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:23 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:40:23.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:23 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:40:24.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:24 smithi096 ceph-mon[31930]: pgmap v361: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:24.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:24 smithi137 ceph-mon[35868]: pgmap v361: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:24 smithi193 ceph-mon[36702]: pgmap v361: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:26.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:25 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:25 smithi137 ceph-mon[35868]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:26.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:25 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:25 smithi193 ceph-mon[36702]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:25 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:26.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:25 smithi096 ceph-mon[31930]: pgmap v362: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-03-18T15:40:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.208+0000 7ff9e11f4640 1 -- 172.21.15.96:0/1362024015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc1022e0 msgr2=0x7ff9dc1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.208+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/1362024015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc1046d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 -- 172.21.15.96:0/1362024015 shutdown_connections 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/1362024015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc107540 0x7ff9dc109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/1362024015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc104c10 0x7ff9dc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/1362024015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc1046d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 -- 172.21.15.96:0/1362024015 >> 172.21.15.96:0/1362024015 conn(0x7ff9dc0fbc70 msgr2=0x7ff9dc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 -- 172.21.15.96:0/1362024015 shutdown_connections 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 -- 172.21.15.96:0/1362024015 wait complete. 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 Processor -- start 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 -- start start 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 0x7ff9dc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.209+0000 7ff9e11f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9e11f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc10d4a0 con 0x7ff9dc107540 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9e11f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc10d320 con 0x7ff9dc1022e0 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9e11f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc10d620 con 0x7ff9dc104c10 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58172/0 (socket says 172.21.15.96:58172) 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9dad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 -- 172.21.15.96:0/4109040659 learned_addr learned my addr 172.21.15.96:0/4109040659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 0x7ff9dc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 -- 172.21.15.96:0/4109040659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 msgr2=0x7ff9dc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 0x7ff9dc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 -- 172.21.15.96:0/4109040659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 msgr2=0x7ff9dc197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 -- 172.21.15.96:0/4109040659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc19f6b0 con 0x7ff9dc107540 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 0x7ff9dc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9db59e640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400eb10 tx=0x7ff9d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9d8d99640 1 -- 172.21.15.96:0/4109040659 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004b20 con 0x7ff9dc107540 2024-03-18T15:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9d8d99640 1 -- 172.21.15.96:0/4109040659 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d4004cc0 con 0x7ff9dc107540 2024-03-18T15:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc19f9a0 con 0x7ff9dc107540 2024-03-18T15:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9d8d99640 1 -- 172.21.15.96:0/4109040659 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4010760 con 0x7ff9dc107540 2024-03-18T15:40:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.210+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc19fef0 con 0x7ff9dc107540 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9d8d99640 1 -- 172.21.15.96:0/4109040659 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7ff9d4005020 con 0x7ff9dc107540 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 Processor -- start 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 -- start start 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d58830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d5b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c08cb8a40 con 0x559c08d53340 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c08c61920 con 0x559c08d53d30 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c08cb4ae0 con 0x559c08d58ea0 2024-03-18T15:40:27.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58178/0 (socket says 172.21.15.96:58178) 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.213+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3081672206 learned_addr learned my addr 172.21.15.96:0/3081672206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d58830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d8d99640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 0x7ff9c80a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d5b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3081672206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 msgr2=0x559c08d5b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d5b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9db59e640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 0x7ff9c80a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3081672206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 msgr2=0x559c08d58830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d58830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3081672206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c08c79460 con 0x559c08d53340 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d8d99640 1 -- 172.21.15.96:0/4109040659 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff9d40cf300 con 0x7ff9dc107540 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40134f0 tx=0x7ff9c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134834b685d8f5af server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d17fa640 1 -- 172.21.15.96:0/3081672206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4022070 con 0x559c08d53340 2024-03-18T15:40:27.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9db59e640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 0x7ff9c80a4e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4007820 tx=0x7ff9d4090ff0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d17fa640 1 -- 172.21.15.96:0/3081672206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c400f940 con 0x559c08d53340 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.214+0000 7ff9d17fa640 1 -- 172.21.15.96:0/3081672206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4003950 con 0x559c08d53340 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3081672206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 msgr2=0x559c08d53760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40134f0 tx=0x7ff9c4009c00 comp rx=0 tx=0).stop 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3081672206 shutdown_connections 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d5b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d58830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3081672206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d53760 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3081672206 >> 172.21.15.96:0/3081672206 conn(0x559c08d4e8b0 msgr2=0x559c08d4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3081672206 shutdown_connections 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3081672206 wait complete. 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 -- start start 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.215+0000 7ff9e3de4740 1 Processor -- start 2024-03-18T15:40:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d5d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d5db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c08d5be50 con 0x559c08d58ea0 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c08c74d70 con 0x559c08d53340 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c08d5bfd0 con 0x559c08d53d30 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 10 client.0 _ll_register_callbacks cb 0x559c089cca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58184/0 (socket says 172.21.15.96:58184) 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3345898902 learned_addr learned my addr 172.21.15.96:0/3345898902 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d5db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d5d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 msgr2=0x559c08d5db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d5db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3345898902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 msgr2=0x559c08d5d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d5d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3345898902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c089cd210 con 0x559c08d58ea0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005780 tx=0x7ff9c4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e618a7db212f06d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4022020 con 0x559c08d58ea0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff9c401dba0 con 0x559c08d58ea0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.216+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c400f940 con 0x559c08d58ea0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9c401dda0 con 0x559c08d58ea0 2024-03-18T15:40:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 10 client.15546 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c08a99620 con 0x559c08d58ea0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c400f940 con 0x559c08d58ea0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c4010430 con 0x559c08d58ea0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 10 client.15546 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 20 client.15546 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08d6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 4 client.15546 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c08cc5f80 con 0x559c08d691e0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.217+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08d6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08d6b5d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a76950a1494ebe server_cookie=e421d921c3e8a453 in_seq=0 out_seq=0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9c27fc640 10 client.15546 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9c27fc640 1 -- 172.21.15.96:0/3345898902 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559c08cc5f80 con 0x559c08d691e0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9c27fc640 10 client.15546 handle_command_reply: tid=0 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9e3de4740 1 client.15546 shutdown 2024-03-18T15:40:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 msgr2=0x559c08d64b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.218+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005780 tx=0x7ff9c4005850 comp rx=0 tx=0).stop 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 shutdown_connections 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08d6b5d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d64b90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d5db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9c27fc640 0 client.15546 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3345898902 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d5d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 >> 172.21.15.96:0/3345898902 conn(0x559c08d4e8b0 msgr2=0x559c08d4dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 shutdown_connections 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3345898902 wait complete. 2024-03-18T15:40:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.219+0000 7ff9e3de4740 20 client.15546 trim_cache size 0 max 16384 2024-03-18T15:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 Processor -- start 2024-03-18T15:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 -- start start 2024-03-18T15:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d52220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a6b80 con 0x559c08d53340 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a86c0 con 0x559c08d53d30 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.234+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a7250 con 0x559c08d58ea0 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58190/0 (socket says 172.21.15.96:58190) 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3363086874 learned_addr learned my addr 172.21.15.96:0/3363086874 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d52220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3363086874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 msgr2=0x559c08d52220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d52220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3363086874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 msgr2=0x559c08d64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d64b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 -- 172.21.15.96:0/3363086874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c08d54ae0 con 0x559c08d53340 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40059f0 tx=0x7ff9c4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dd4dbba4c213aa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d0ff9640 1 -- 172.21.15.96:0/3363086874 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4022040 con 0x559c08d53340 2024-03-18T15:40:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d0ff9640 1 -- 172.21.15.96:0/3363086874 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4005590 con 0x559c08d53340 2024-03-18T15:40:27.238 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.235+0000 7ff9d0ff9640 1 -- 172.21.15.96:0/3363086874 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4010430 con 0x559c08d53340 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3363086874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 msgr2=0x559c08d4e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40059f0 tx=0x7ff9c4009310 comp rx=0 tx=0).stop 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3363086874 shutdown_connections 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d58ea0 0x559c08d52220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53d30 0x559c08d64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/3363086874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d53340 0x559c08d4e330 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.236+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3363086874 >> 172.21.15.96:0/3363086874 conn(0x559c08d4e8b0 msgr2=0x559c08d51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3363086874 shutdown_connections 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- 172.21.15.96:0/3363086874 wait complete. 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- start start 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 Processor -- start 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d61e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d62360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a7ff0 con 0x559c08d58ea0 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a7920 con 0x559c08d53340 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc1a4b60 con 0x559c08d53d30 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9e3de4740 10 client.0 _ll_register_callbacks cb 0x7ff9c4020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58196/0 (socket says 172.21.15.96:58196) 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 -- 172.21.15.96:0/663008458 learned_addr learned my addr 172.21.15.96:0/663008458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d62360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d61e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 -- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 msgr2=0x559c08d62360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d62360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 -- 172.21.15.96:0/663008458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 msgr2=0x559c08d61e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.237+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d61e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9d3fff640 1 -- 172.21.15.96:0/663008458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c08d63960 con 0x559c08d58ea0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9d3fff640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4013520 tx=0x7ff9c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c2648b3ba4d71b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4009ea0 con 0x559c08d58ea0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c4005590 con 0x559c08d58ea0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff9c401dba0 con 0x559c08d58ea0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9c4013780 con 0x559c08d58ea0 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 10 client.15552 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c08d64cc0 con 0x559c08d58ea0 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4005590 con 0x559c08d58ea0 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff9c4003900 con 0x559c08d58ea0 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 10 client.15552 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.238+0000 7ff9e3de4740 20 client.15552 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08dd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9e3de4740 4 client.15552 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c08cc02d0 con 0x559c08d691e0 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08dd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9c27fc640 10 client.15552 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.239+0000 7ff9dad9d640 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08dd19c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d172d806cfc16e5b server_cookie=25d90560fe7cdb29 in_seq=0 out_seq=0 2024-03-18T15:40:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.249+0000 7ff9c27fc640 1 -- 172.21.15.96:0/663008458 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c08cc02d0 con 0x559c08d691e0 2024-03-18T15:40:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.249+0000 7ff9c27fc640 10 client.15552 handle_command_reply: tid=0 2024-03-18T15:40:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.249+0000 7ff9e3de4740 1 client.15552 shutdown 2024-03-18T15:40:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 msgr2=0x559c08d63370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4013520 tx=0x7ff9c4005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 shutdown_connections 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c08d691e0 0x559c08dd19c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c08d58ea0 0x559c08d63370 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c08d53d30 0x559c08d62360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 --2- 172.21.15.96:0/663008458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c08d53340 0x559c08d61e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 >> 172.21.15.96:0/663008458 conn(0x559c08d4e8b0 msgr2=0x559c08d51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9c27fc640 0 client.15552 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 shutdown_connections 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 1 -- 172.21.15.96:0/663008458 wait complete. 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e3de4740 20 client.15552 trim_cache size 0 max 16384 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:40:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 msgr2=0x7ff9c80a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 0x7ff9c80a4e00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4007820 tx=0x7ff9d4090ff0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 msgr2=0x7ff9dc19f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400eb10 tx=0x7ff9d400c2d0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 shutdown_connections 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff9c80a2940 0x7ff9c80a4e00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9dc107540 0x7ff9dc19f040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9dc104c10 0x7ff9dc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 --2- 172.21.15.96:0/4109040659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9dc1022e0 0x7ff9dc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.250+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 >> 172.21.15.96:0/4109040659 conn(0x7ff9dc0fbc70 msgr2=0x7ff9dc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.251+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 shutdown_connections 2024-03-18T15:40:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:27.251+0000 7ff9e11f4640 1 -- 172.21.15.96:0/4109040659 wait complete. 2024-03-18T15:40:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:40:27.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:40:27.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:40:27.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:40:27.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:40:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:40:27.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:40:28.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:28 smithi137 ceph-mon[35868]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T15:40:28.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:28 smithi193 ceph-mon[36702]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T15:40:28.436 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:28 smithi096 ceph-mon[31930]: pgmap v363: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-18T15:40:30.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:30 smithi137 ceph-mon[35868]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-18T15:40:30.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:30 smithi193 ceph-mon[36702]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-18T15:40:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:30 smithi096 ceph-mon[31930]: pgmap v364: 97 pgs: 97 active+clean; 658 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-03-18T15:40:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:32 smithi137 ceph-mon[35868]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-18T15:40:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:32 smithi193 ceph-mon[36702]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-18T15:40:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:32 smithi096 ceph-mon[31930]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-18T15:40:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:34 smithi137 ceph-mon[35868]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:34.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:34 smithi193 ceph-mon[36702]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:34 smithi096 ceph-mon[31930]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:36 smithi137 ceph-mon[35868]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-18T15:40:36.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:36 smithi193 ceph-mon[36702]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-18T15:40:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:36 smithi096 ceph-mon[31930]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 15 op/s 2024-03-18T15:40:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:38 smithi137 ceph-mon[35868]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-18T15:40:38.954 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:38 smithi096 ceph-mon[31930]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-18T15:40:38.954 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:38 smithi193 ceph-mon[36702]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-18T15:40:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 -- 172.21.15.96:0/631837931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c100700 msgr2=0x7f593c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 --2- 172.21.15.96:0/631837931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c100700 0x7f593c100b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f592c009860 tx=0x7f592c02bb70 comp rx=0 tx=0).stop 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 -- 172.21.15.96:0/631837931 shutdown_connections 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 --2- 172.21.15.96:0/631837931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1022b0 0x7f593c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 --2- 172.21.15.96:0/631837931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1018f0 0x7f593c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 --2- 172.21.15.96:0/631837931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c100700 0x7f593c100b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 -- 172.21.15.96:0/631837931 >> 172.21.15.96:0/631837931 conn(0x7f593c0fbc70 msgr2=0x7f593c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 -- 172.21.15.96:0/631837931 shutdown_connections 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.381+0000 7f5941537640 1 -- 172.21.15.96:0/631837931 wait complete. 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.382+0000 7f5941537640 1 Processor -- start 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.382+0000 7f5941537640 1 -- start start 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.382+0000 7f5941537640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5941537640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 0x7f593c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5941537640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 0x7f593c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5941537640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10c350 con 0x7f593c1022b0 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5941537640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10c1d0 con 0x7f593c1018f0 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5941537640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10c4d0 con 0x7f593c100700 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 0x7f593c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:50444/0 (socket says 172.21.15.96:50444) 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 0x7f593c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57550/0 (socket says 172.21.15.96:57550) 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 -- 172.21.15.96:0/2568789154 learned_addr learned my addr 172.21.15.96:0/2568789154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593a7fc640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 0x7f593c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 -- 172.21.15.96:0/2568789154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 msgr2=0x7f593c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 0x7f593c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 -- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 msgr2=0x7f593c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 0x7f593c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 -- 172.21.15.96:0/2568789154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19f6f0 con 0x7f593c100700 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593a7fc640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 0x7f593c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f593affd640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f592c0349e0 tx=0x7f592c032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5923fff640 1 -- 172.21.15.96:0/2568789154 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c041020 con 0x7f593c100700 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.383+0000 7f5923fff640 1 -- 172.21.15.96:0/2568789154 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c02c9f0 con 0x7f593c100700 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.384+0000 7f5923fff640 1 -- 172.21.15.96:0/2568789154 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c039040 con 0x7f593c100700 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.384+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c102b10 con 0x7f593c100700 2024-03-18T15:40:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.384+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c102fb0 con 0x7f593c100700 2024-03-18T15:40:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.387+0000 7f5923fff640 1 -- 172.21.15.96:0/2568789154 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f592c044020 con 0x7f593c100700 2024-03-18T15:40:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 Processor -- start 2024-03-18T15:40:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 -- start start 2024-03-18T15:40:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317340 0x55ab6c317760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 0x55ab6c31c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c27ca40 con 0x55ab6c317d30 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c225920 con 0x55ab6c31cea0 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f59441f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c278ae0 con 0x55ab6c317340 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f5923fff640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 0x7f59240a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:47564/0 (socket says 172.21.15.96:47564) 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f5923fff640 1 -- 172.21.15.96:0/2568789154 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f592c0efab0 con 0x7f593c100700 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 0x55ab6c31c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f593a7fc640 1 -- 172.21.15.96:0/3562985887 learned_addr learned my addr 172.21.15.96:0/3562985887 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.388+0000 7f593b7fe640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 0x7f59240a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 -- 172.21.15.96:0/3562985887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317340 msgr2=0x55ab6c317760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317340 0x55ab6c317760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 -- 172.21.15.96:0/3562985887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 msgr2=0x55ab6c31c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 0x55ab6c31c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 -- 172.21.15.96:0/3562985887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6c23d460 con 0x55ab6c31cea0 2024-03-18T15:40:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593affd640 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 0x55ab6c31c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593a7fc640 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5930009c00 tx=0x7f5930010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0e834d3f6400e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f593b7fe640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 0x7f59240a4e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5934005fd0 tx=0x7f5934010cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f59217fa640 1 -- 172.21.15.96:0/3562985887 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930010e60 con 0x55ab6c31cea0 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.389+0000 7f59217fa640 1 -- 172.21.15.96:0/3562985887 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5930005500 con 0x55ab6c31cea0 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.390+0000 7f59217fa640 1 -- 172.21.15.96:0/3562985887 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930012690 con 0x55ab6c31cea0 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- 172.21.15.96:0/3562985887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 msgr2=0x55ab6c31f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5930009c00 tx=0x7f5930010370 comp rx=0 tx=0).stop 2024-03-18T15:40:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- 172.21.15.96:0/3562985887 shutdown_connections 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c31f330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317d30 0x55ab6c31c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 --2- 172.21.15.96:0/3562985887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317340 0x55ab6c317760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- 172.21.15.96:0/3562985887 >> 172.21.15.96:0/3562985887 conn(0x55ab6c3128b0 msgr2=0x55ab6c312cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- 172.21.15.96:0/3562985887 shutdown_connections 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- 172.21.15.96:0/3562985887 wait complete. 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 -- start start 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.391+0000 7f59441f8740 1 Processor -- start 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c321640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c31fef0 con 0x55ab6c317340 2024-03-18T15:40:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c238d70 con 0x55ab6c317d30 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ab6c320070 con 0x55ab6c31cea0 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f59441f8740 10 client.0 _ll_register_callbacks cb 0x55ab6bf90a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c321640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:47566/0 (socket says 172.21.15.96:47566) 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:50462/0 (socket says 172.21.15.96:50462) 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 -- 172.21.15.96:0/1838622425 learned_addr learned my addr 172.21.15.96:0/1838622425 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 -- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 msgr2=0x55ab6c328bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 -- 172.21.15.96:0/1838622425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 msgr2=0x55ab6c321640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c321640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593a7fc640 1 -- 172.21.15.96:0/1838622425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6bf91210 con 0x55ab6c317d30 2024-03-18T15:40:40.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593b7fe640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.392+0000 7f593affd640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c321640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f593a7fc640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f55623c200b441 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930004220 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f59300043c0 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5930005500 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5930005340 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f59441f8740 10 client.25355 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab6c05d620 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5930005500 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.393+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f593000f040 con 0x55ab6c317d30 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 10 client.25355 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:40.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 20 client.25355 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c32d160 0x55ab6c32f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 4 client.25355 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6c289f80 con 0x55ab6c32d160 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f593affd640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c32d160 0x55ab6c32f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f591eff5640 10 client.25355 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.394+0000 7f593affd640 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c32d160 0x55ab6c32f620 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec9421de303de7f server_cookie=da3a7b768218c417 in_seq=0 out_seq=0 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.395+0000 7f591eff5640 1 -- 172.21.15.96:0/1838622425 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ab6c289f80 con 0x55ab6c32d160 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.395+0000 7f591eff5640 10 client.25355 handle_command_reply: tid=0 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.395+0000 7f59441f8740 1 client.25355 shutdown 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.395+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 msgr2=0x55ab6c321b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.395+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5930000c00 tx=0x7f5930005760 comp rx=0 tx=0).stop 2024-03-18T15:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 shutdown_connections 2024-03-18T15:40:40.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c31cea0 0x55ab6c328bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c317d30 0x55ab6c321b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c32d160 0x55ab6c32f620 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 --2- 172.21.15.96:0/1838622425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c321640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f591eff5640 0 client.25355 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 >> 172.21.15.96:0/1838622425 conn(0x55ab6c3128b0 msgr2=0x55ab6c311bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 shutdown_connections 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 1 -- 172.21.15.96:0/1838622425 wait complete. 2024-03-18T15:40:40.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.396+0000 7f59441f8740 20 client.25355 trim_cache size 0 max 16384 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 Processor -- start 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 -- start start 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c328b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c30da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a6b60 con 0x55ab6c317340 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a86a0 con 0x55ab6c31cea0 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f59441f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a7230 con 0x55ab6c317d30 2024-03-18T15:40:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57570/0 (socket says 172.21.15.96:57570) 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593affd640 1 -- 172.21.15.96:0/126941427 learned_addr learned my addr 172.21.15.96:0/126941427 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593b7fe640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c30da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593a7fc640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c328b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.425+0000 7f593affd640 1 -- 172.21.15.96:0/126941427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 msgr2=0x55ab6c328b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593affd640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c328b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593affd640 1 -- 172.21.15.96:0/126941427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 msgr2=0x55ab6c30da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593affd640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c30da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593affd640 1 -- 172.21.15.96:0/126941427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6c3201f0 con 0x55ab6c317340 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593a7fc640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c328b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f593affd640 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f592c0b8d40 tx=0x7f592c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910c02e492f623e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f5920ff9640 1 -- 172.21.15.96:0/126941427 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0bca70 con 0x55ab6c317340 2024-03-18T15:40:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f5920ff9640 1 -- 172.21.15.96:0/126941427 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c0bc370 con 0x55ab6c317340 2024-03-18T15:40:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.426+0000 7f5920ff9640 1 -- 172.21.15.96:0/126941427 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c039270 con 0x55ab6c317340 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- 172.21.15.96:0/126941427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 msgr2=0x55ab6c31f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f592c0b8d40 tx=0x7f592c0dc040 comp rx=0 tx=0).stop 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- 172.21.15.96:0/126941427 shutdown_connections 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c30da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c328b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 --2- 172.21.15.96:0/126941427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c31f290 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- 172.21.15.96:0/126941427 >> 172.21.15.96:0/126941427 conn(0x55ab6c3128b0 msgr2=0x55ab6c312c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- 172.21.15.96:0/126941427 shutdown_connections 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- 172.21.15.96:0/126941427 wait complete. 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.427+0000 7f59441f8740 1 -- start start 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 Processor -- start 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f593affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57580/0 (socket says 172.21.15.96:57580) 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f593affd640 1 -- 172.21.15.96:0/2250099854 learned_addr learned my addr 172.21.15.96:0/2250099854 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c325dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c391510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a7fd0 con 0x55ab6c317340 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a7900 con 0x55ab6c31cea0 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f593c1a4b40 con 0x55ab6c317d30 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.428+0000 7f593a7fc640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c325dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593b7fe640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c391510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f59441f8740 10 client.0 _ll_register_callbacks cb 0x7f5930009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 -- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 msgr2=0x55ab6c325dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c325dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 -- 172.21.15.96:0/2250099854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 msgr2=0x55ab6c391510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c391510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6c3263e0 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c0bc370 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f593affd640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f592c005b60 tx=0x7f592c0ba880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4204a6f276c76244 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f591eff5640 10 client.15570 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f591eff5640 1 -- 172.21.15.96:0/2250099854 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0d4030 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.429+0000 7f591eff5640 1 -- 172.21.15.96:0/2250099854 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f592c02cba0 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f591eff5640 1 -- 172.21.15.96:0/2250099854 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f592c0c4b10 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 10 client.15570 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55ab6c392a00 con 0x55ab6c317340 2024-03-18T15:40:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f591eff5640 1 -- 172.21.15.96:0/2250099854 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f592c03c030 con 0x55ab6c317340 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 10 client.15570 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 20 client.15570 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c393240 0x55ab6c395660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 4 client.15570 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab6c2842d0 con 0x55ab6c393240 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.430+0000 7f593a7fc640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c393240 0x55ab6c395660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.431+0000 7f591eff5640 10 client.15570 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.431+0000 7f593a7fc640 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c393240 0x55ab6c395660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b66181249d5e19c server_cookie=aa8468ca8be8a622 in_seq=0 out_seq=0 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f591eff5640 1 -- 172.21.15.96:0/2250099854 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab6c2842d0 con 0x55ab6c393240 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f591eff5640 10 client.15570 handle_command_reply: tid=0 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 client.15570 shutdown 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 msgr2=0x55ab6c325890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f592c005b60 tx=0x7f592c0ba880 comp rx=0 tx=0).stop 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 shutdown_connections 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab6c31cea0 0x55ab6c391510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ab6c317d30 0x55ab6c325dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55ab6c393240 0x55ab6c395660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f591eff5640 0 client.15570 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 --2- 172.21.15.96:0/2250099854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6c317340 0x55ab6c325890 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 >> 172.21.15.96:0/2250099854 conn(0x55ab6c3128b0 msgr2=0x55ab6c30da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 shutdown_connections 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 1 -- 172.21.15.96:0/2250099854 wait complete. 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.434+0000 7f59441f8740 20 client.15570 trim_cache size 0 max 16384 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:40:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:40:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 msgr2=0x7f59240a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 0x7f59240a4e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5934005fd0 tx=0x7f5934010cf0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 msgr2=0x7f593c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f592c0349e0 tx=0x7f592c032430 comp rx=0 tx=0).stop 2024-03-18T15:40:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 shutdown_connections 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59240a29c0 0x7f59240a4e80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1022b0 0x7f593c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1018f0 0x7f593c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 --2- 172.21.15.96:0/2568789154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c100700 0x7f593c197b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 >> 172.21.15.96:0/2568789154 conn(0x7f593c0fbc70 msgr2=0x7f593c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 shutdown_connections 2024-03-18T15:40:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:40.435+0000 7f5941537640 1 -- 172.21.15.96:0/2568789154 wait complete. 2024-03-18T15:40:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:40 smithi137 ceph-mon[35868]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:40.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:40 smithi193 ceph-mon[36702]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:40 smithi096 ceph-mon[31930]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-03-18T15:40:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:42 smithi137 ceph-mon[35868]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-18T15:40:42.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:42 smithi193 ceph-mon[36702]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-18T15:40:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:42 smithi096 ceph-mon[31930]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 938 B/s wr, 12 op/s 2024-03-18T15:40:44.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:40:44.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:40:44.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 -- 172.21.15.96:0/183658735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 msgr2=0x7f3064100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 --2- 172.21.15.96:0/183658735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064100b00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3060009ca0 tx=0x7f306002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 -- 172.21.15.96:0/183658735 shutdown_connections 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 --2- 172.21.15.96:0/183658735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3064102290 0x7f3064108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 --2- 172.21.15.96:0/183658735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30641018d0 0x7f3064101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 --2- 172.21.15.96:0/183658735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064100b00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 -- 172.21.15.96:0/183658735 >> 172.21.15.96:0/183658735 conn(0x7f30640fbc70 msgr2=0x7f30640fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 -- 172.21.15.96:0/183658735 shutdown_connections 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.567+0000 7f306b376640 1 -- 172.21.15.96:0/183658735 wait complete. 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 Processor -- start 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 -- start start 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 0x7f3064196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 0x7f306419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f306410c350 con 0x7f30641006e0 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f306410c1d0 con 0x7f30641018d0 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f306b376640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f306410c4d0 con 0x7f3064102290 2024-03-18T15:40:44.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53950/0 (socket says 172.21.15.96:53950) 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f3069112640 1 -- 172.21.15.96:0/166545969 learned_addr learned my addr 172.21.15.96:0/166545969 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f3068911640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 0x7f3064196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.568+0000 7f3069913640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 0x7f306419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 -- 172.21.15.96:0/166545969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 msgr2=0x7f306419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 0x7f306419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 -- 172.21.15.96:0/166545969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 msgr2=0x7f3064196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 0x7f3064196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 -- 172.21.15.96:0/166545969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306419e670 con 0x7f30641006e0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3068911640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 0x7f3064196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069913640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 0x7f306419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f3069112640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3060037040 tx=0x7f3060032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f305a7fc640 1 -- 172.21.15.96:0/166545969 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600346c0 con 0x7f30641006e0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3064068940 con 0x7f30641006e0 2024-03-18T15:40:44.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f305a7fc640 1 -- 172.21.15.96:0/166545969 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3060032690 con 0x7f30641006e0 2024-03-18T15:40:44.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f305a7fc640 1 -- 172.21.15.96:0/166545969 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600042a0 con 0x7f30641006e0 2024-03-18T15:40:44.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.569+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3064068e80 con 0x7f30641006e0 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f305a7fc640 1 -- 172.21.15.96:0/166545969 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3060032840 con 0x7f30641006e0 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 Processor -- start 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 -- start start 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb3829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb38a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cb2e3500 con 0x5631cb3880e0 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cb28bbe0 con 0x5631cb382f70 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f306e033740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cb2df5a0 con 0x5631cb382580 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3069112640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3069913640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb3829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3069112640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51236/0 (socket says 172.21.15.96:51236) 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3069112640 1 -- 172.21.15.96:0/1179209250 learned_addr learned my addr 172.21.15.96:0/1179209250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3068911640 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb38a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f305a7fc640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 0x7f30500a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.573+0000 7f3069112640 1 -- 172.21.15.96:0/1179209250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 msgr2=0x5631cb3829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069112640 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb3829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069112640 1 -- 172.21.15.96:0/1179209250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 msgr2=0x5631cb38a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069112640 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb38a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069112640 1 -- 172.21.15.96:0/1179209250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cb29f4b0 con 0x5631cb382f70 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069913640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 0x7f30500a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f305a7fc640 1 -- 172.21.15.96:0/166545969 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f30600f2940 con 0x7f30641006e0 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069112640 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30600f0310 tx=0x7f30600b47d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbb1e6898a84e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.574+0000 7f3069913640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 0x7f30500a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5631cb3837d0 tx=0x7f305c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f304f7fe640 1 -- 172.21.15.96:0/1179209250 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600b9e60 con 0x5631cb382f70 2024-03-18T15:40:44.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f304f7fe640 1 -- 172.21.15.96:0/1179209250 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306008a510 con 0x5631cb382f70 2024-03-18T15:40:44.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f304f7fe640 1 -- 172.21.15.96:0/1179209250 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306008e1c0 con 0x5631cb382f70 2024-03-18T15:40:44.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f306e033740 1 -- 172.21.15.96:0/1179209250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 msgr2=0x5631cb387a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f306e033740 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30600f0310 tx=0x7f30600b47d0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f306e033740 1 -- 172.21.15.96:0/1179209250 shutdown_connections 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f306e033740 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb38a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.585+0000 7f306e033740 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb387a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 --2- 172.21.15.96:0/1179209250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb3829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- 172.21.15.96:0/1179209250 >> 172.21.15.96:0/1179209250 conn(0x5631cb3796e0 msgr2=0x5631cb379ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- 172.21.15.96:0/1179209250 shutdown_connections 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- 172.21.15.96:0/1179209250 wait complete. 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- start start 2024-03-18T15:40:44.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 Processor -- start 2024-03-18T15:40:44.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb38cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb393d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5631cb38ae50 con 0x5631cb3880e0 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5631cb29f160 con 0x5631cb382f70 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.586+0000 7f306e033740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631cb38afd0 con 0x5631cb382580 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f306e033740 10 client.0 _ll_register_callbacks cb 0x5631cb283150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39606/0 (socket says 172.21.15.96:39606) 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 -- 172.21.15.96:0/2417430154 learned_addr learned my addr 172.21.15.96:0/2417430154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069913640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb393d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3068911640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb38cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 -- 172.21.15.96:0/2417430154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 msgr2=0x5631cb38cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb38cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 -- 172.21.15.96:0/2417430154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 msgr2=0x5631cb393d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb393d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 -- 172.21.15.96:0/2417430154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cb0c17d0 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f3069112640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30600bf280 tx=0x7f30600f0370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a4e693a1a9d4b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f304cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600b9e60 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f306008a510 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f30600bd5f0 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f30600bf9f0 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f306e033740 10 client.25429 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.587+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631caff73f0 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306008a510 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30600f0480 con 0x5631cb382580 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 10 client.25429 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 10 client.25429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 20 client.25429 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:44.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3981c0 0x5631cb39a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 4 client.25429 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631cb2f0a40 con 0x5631cb3981c0 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.588+0000 7f3068911640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3981c0 0x5631cb39a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f304cff9640 10 client.25429 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f3068911640 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3981c0 0x5631cb39a5f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d73485d215b33cd server_cookie=9d1d13324b3cd0cd in_seq=0 out_seq=0 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f304cff9640 1 -- 172.21.15.96:0/2417430154 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5631cb2f0a40 con 0x5631cb3981c0 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f304cff9640 10 client.25429 handle_command_reply: tid=0 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 client.25429 shutdown 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 msgr2=0x5631cb38c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f30600bf280 tx=0x7f30600f0370 comp rx=0 tx=0).stop 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 shutdown_connections 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3981c0 0x5631cb39a5f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb3880e0 0x5631cb393d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f304cff9640 0 client.25429 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb382f70 0x5631cb38cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 --2- 172.21.15.96:0/2417430154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382580 0x5631cb38c7c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.589+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 >> 172.21.15.96:0/2417430154 conn(0x5631cb3796e0 msgr2=0x5631cb378900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.590+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 shutdown_connections 2024-03-18T15:40:44.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.590+0000 7f306e033740 1 -- 172.21.15.96:0/2417430154 wait complete. 2024-03-18T15:40:44.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.590+0000 7f306e033740 20 client.25429 trim_cache size 0 max 16384 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 Processor -- start 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 -- start start 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb393cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb378070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a6bf0 con 0x5631cb382580 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a8730 con 0x5631cb3880e0 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f306e033740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a72c0 con 0x5631cb382f70 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53968/0 (socket says 172.21.15.96:53968) 2024-03-18T15:40:44.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3068911640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb393cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3069112640 1 -- 172.21.15.96:0/3953744120 learned_addr learned my addr 172.21.15.96:0/3953744120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3069913640 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb378070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.607+0000 7f3068911640 1 -- 172.21.15.96:0/3953744120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 msgr2=0x5631cb378070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3068911640 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb378070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3068911640 1 -- 172.21.15.96:0/3953744120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 msgr2=0x5631cb38cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3068911640 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3068911640 1 -- 172.21.15.96:0/3953744120 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cafb70f0 con 0x5631cb382f70 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3069112640 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f3068911640 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb393cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5631cb39ae70 tx=0x7f3054005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1d83c540f64367d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f304effd640 1 -- 172.21.15.96:0/3953744120 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054022070 con 0x5631cb382f70 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f304effd640 1 -- 172.21.15.96:0/3953744120 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305401ed90 con 0x5631cb382f70 2024-03-18T15:40:44.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.608+0000 7f304effd640 1 -- 172.21.15.96:0/3953744120 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054004a90 con 0x5631cb382f70 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- 172.21.15.96:0/3953744120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 msgr2=0x5631cb393cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb393cd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5631cb39ae70 tx=0x7f3054005f90 comp rx=0 tx=0).stop 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- 172.21.15.96:0/3953744120 shutdown_connections 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb378070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb393cd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- 172.21.15.96:0/3953744120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb38cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- 172.21.15.96:0/3953744120 >> 172.21.15.96:0/3953744120 conn(0x5631cb3796e0 msgr2=0x5631cb38c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- 172.21.15.96:0/3953744120 shutdown_connections 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- 172.21.15.96:0/3953744120 wait complete. 2024-03-18T15:40:44.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 -- start start 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 Processor -- start 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb391540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.609+0000 7f306e033740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb392550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f306e033740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a8060 con 0x5631cb382580 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f306e033740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a7990 con 0x5631cb3880e0 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f306e033740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30641a4bd0 con 0x5631cb382f70 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f306e033740 10 client.0 _ll_register_callbacks cb 0x7f30540095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53972/0 (socket says 172.21.15.96:53972) 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 -- 172.21.15.96:0/719918742 learned_addr learned my addr 172.21.15.96:0/719918742 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3068911640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb391540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069913640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb392550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 -- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 msgr2=0x5631cb391540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb391540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 -- 172.21.15.96:0/719918742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 msgr2=0x5631cb392550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb392550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 -- 172.21.15.96:0/719918742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cb2aa9d0 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f3069112640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30600bb750 tx=0x7f30600c3330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d79268083e55e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f304cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600b9e60 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f30600df0d0 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.610+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f305401ed90 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3060039050 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 10 client.15588 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631cb2aa9d0 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f305401ed90 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f30600f0450 con 0x5631cb382580 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 10 client.15588 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:44.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 20 client.15588 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3fe5f0 0x5631cb4009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 4 client.15588 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631cb2ead90 con 0x5631cb3fe5f0 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.611+0000 7f3068911640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3fe5f0 0x5631cb4009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f304cff9640 10 client.15588 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f3068911640 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3fe5f0 0x5631cb4009e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe88e93fa30be3a server_cookie=28ded2e95656df92 in_seq=0 out_seq=0 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f304cff9640 1 -- 172.21.15.96:0/719918742 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631cb2ead90 con 0x5631cb3fe5f0 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f304cff9640 10 client.15588 handle_command_reply: tid=0 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 client.15588 shutdown 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 msgr2=0x5631cb391000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30600bb750 tx=0x7f30600c3330 comp rx=0 tx=0).stop 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 shutdown_connections 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5631cb3880e0 0x5631cb392550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631cb382f70 0x5631cb391540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5631cb3fe5f0 0x5631cb4009e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f304cff9640 0 client.15588 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 --2- 172.21.15.96:0/719918742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631cb382580 0x5631cb391000 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 >> 172.21.15.96:0/719918742 conn(0x5631cb3796e0 msgr2=0x5631cb37c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 shutdown_connections 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 1 -- 172.21.15.96:0/719918742 wait complete. 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.612+0000 7f306e033740 20 client.15588 trim_cache size 0 max 16384 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:40:44.619 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:40:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 msgr2=0x7f30500a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 0x7f30500a4d50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5631cb3837d0 tx=0x7f305c0046c0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 msgr2=0x7f3064196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3060037040 tx=0x7f3060032f40 comp rx=0 tx=0).stop 2024-03-18T15:40:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 shutdown_connections 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30500a2890 0x7f30500a4d50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3064102290 0x7f306419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30641018d0 0x7f3064196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 --2- 172.21.15.96:0/166545969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30641006e0 0x7f3064196a50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 >> 172.21.15.96:0/166545969 conn(0x7f30640fbc70 msgr2=0x7f3064104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 shutdown_connections 2024-03-18T15:40:44.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.613+0000 7f306b376640 1 -- 172.21.15.96:0/166545969 wait complete. 2024-03-18T15:40:44.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:40:44.626 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 -- 172.21.15.96:0/2234915863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec0100720 msgr2=0x7f5ec0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/2234915863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec0100720 0x7f5ec0100b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009860 tx=0x7f5ebc02bb70 comp rx=0 tx=0).stop 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 -- 172.21.15.96:0/2234915863 shutdown_connections 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/2234915863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec01022d0 0x7f5ec0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/2234915863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0101910 0x7f5ec0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/2234915863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec0100720 0x7f5ec0100b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 -- 172.21.15.96:0/2234915863 >> 172.21.15.96:0/2234915863 conn(0x7f5ec00fbc90 msgr2=0x7f5ec00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:44.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 -- 172.21.15.96:0/2234915863 shutdown_connections 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.732+0000 7f5ec7a97640 1 -- 172.21.15.96:0/2234915863 wait complete. 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 Processor -- start 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 -- start start 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0100720 0x7f5ec010b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec01022d0 0x7f5ec0109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010d4c0 con 0x7f5ec0101910 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010d340 con 0x7f5ec01022d0 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec7a97640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec010d640 con 0x7f5ec0100720 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53986/0 (socket says 172.21.15.96:53986) 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 -- 172.21.15.96:0/4257473599 learned_addr learned my addr 172.21.15.96:0/4257473599 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5833640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0100720 0x7f5ec010b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec6034640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec01022d0 0x7f5ec0109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 -- 172.21.15.96:0/4257473599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0100720 msgr2=0x7f5ec010b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0100720 0x7f5ec010b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 -- 172.21.15.96:0/4257473599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec01022d0 msgr2=0x7f5ec0109c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec01022d0 0x7f5ec0109c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.733+0000 7f5ec5032640 1 -- 172.21.15.96:0/4257473599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec010a4b0 con 0x7f5ec0101910 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5ec5032640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9f0 tx=0x7f5eb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb000cd80 con 0x7f5ec0101910 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb0005500 con 0x7f5ec0101910 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec01a1d00 con 0x7f5ec0101910 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0010690 con 0x7f5ec0101910 2024-03-18T15:40:44.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.734+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec01a21a0 con 0x7f5ec0101910 2024-03-18T15:40:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.735+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec0107ac0 con 0x7f5ec0101910 2024-03-18T15:40:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.736+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f5eb00046c0 con 0x7f5ec0101910 2024-03-18T15:40:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.737+0000 7f5eb6ffd640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 0x7f5eac0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.737+0000 7f5ec5833640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 0x7f5eac0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:44.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.737+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5eb00cf040 con 0x7f5ec0101910 2024-03-18T15:40:44.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.738+0000 7f5ec5833640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 0x7f5eac0a4e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc02c180 tx=0x7f5ebc034ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:44.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:44.742+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f5eb009bef0 con 0x7f5ec0101910 2024-03-18T15:40:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:44 smithi137 ceph-mon[35868]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:40:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:44 smithi193 ceph-mon[36702]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:40:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:44 smithi096 ceph-mon[31930]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-18T15:40:45.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.099+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ec002d1b0 con 0x7f5ec0101910 2024-03-18T15:40:45.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.100+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/4257473599 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7f5eb009b7f0 con 0x7f5ec0101910 2024-03-18T15:40:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:40:45.101 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:40:03.325244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:40:45.102 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-03-18T15:40:45.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 msgr2=0x7f5eac0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 0x7f5eac0a4e50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc02c180 tx=0x7f5ebc034ad0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 msgr2=0x7f5ec01096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000e9f0 tx=0x7f5eb000edf0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 shutdown_connections 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5eac0a2990 0x7f5eac0a4e50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ec01022d0 0x7f5ec0109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ec0101910 0x7f5ec01096e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 --2- 172.21.15.96:0/4257473599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ec0100720 0x7f5ec010b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.102+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 >> 172.21.15.96:0/4257473599 conn(0x7f5ec00fbc90 msgr2=0x7f5ec0104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.103+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 shutdown_connections 2024-03-18T15:40:45.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.103+0000 7f5ec7a97640 1 -- 172.21.15.96:0/4257473599 wait complete. 2024-03-18T15:40:45.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:40:45.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.219+0000 7fc78edf4640 1 -- 172.21.15.96:0/675719057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881033a0 msgr2=0x7fc788109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.219+0000 7fc78edf4640 1 --2- 172.21.15.96:0/675719057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881033a0 0x7fc788109c30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc780009be0 tx=0x7fc78002bef0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.219+0000 7fc78edf4640 1 -- 172.21.15.96:0/675719057 shutdown_connections 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.219+0000 7fc78edf4640 1 --2- 172.21.15.96:0/675719057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881033a0 0x7fc788109c30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 --2- 172.21.15.96:0/675719057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881029e0 0x7fc788102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 --2- 172.21.15.96:0/675719057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881017f0 0x7fc788101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 -- 172.21.15.96:0/675719057 >> 172.21.15.96:0/675719057 conn(0x7fc7880fcd60 msgr2=0x7fc7880ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 -- 172.21.15.96:0/675719057 shutdown_connections 2024-03-18T15:40:45.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 -- 172.21.15.96:0/675719057 wait complete. 2024-03-18T15:40:45.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 Processor -- start 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.220+0000 7fc78edf4640 1 -- start start 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 0x7fc788197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78cb90640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 0x7fc788197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc787fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78cb90640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 0x7fc788197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51306/0 (socket says 172.21.15.96:51306) 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc787fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39686/0 (socket says 172.21.15.96:39686) 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78d391640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78d391640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53998/0 (socket says 172.21.15.96:53998) 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78cb90640 1 -- 172.21.15.96:0/3100384836 learned_addr learned my addr 172.21.15.96:0/3100384836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d470 con 0x7fc7881033a0 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d2f0 con 0x7fc7881017f0 2024-03-18T15:40:45.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.221+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d5f0 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 -- 172.21.15.96:0/3100384836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 msgr2=0x7fc788197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 0x7fc788197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 -- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 msgr2=0x7fc78819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78819f6e0 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc78d391640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc787fff640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc77400e9f0 tx=0x7fc77400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc785ffb640 1 -- 172.21.15.96:0/3100384836 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77400ccc0 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78819f9d0 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc785ffb640 1 -- 172.21.15.96:0/3100384836 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc774005020 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc785ffb640 1 -- 172.21.15.96:0/3100384836 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740049e0 con 0x7fc7881029e0 2024-03-18T15:40:45.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.222+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78819ff20 con 0x7fc7881029e0 2024-03-18T15:40:45.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc7919e4740 1 Processor -- start 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc7919e4740 1 -- start start 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253d9bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc785ffb640 1 -- 172.21.15.96:0/3100384836 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fc774005560 con 0x7fc7881029e0 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc78d391640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc78d391640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39702/0 (socket says 172.21.15.96:39702) 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.225+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253d9e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc78d391640 1 -- 172.21.15.96:0/1491974540 learned_addr learned my addr 172.21.15.96:0/1491974540 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a253cfb490 con 0x55a253d9c3b0 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a253ca3c90 con 0x55a253d97240 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a253cf7530 con 0x55a253d96850 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc78cb90640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253d9bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253d9e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc785ffb640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 0x7fc7780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 -- 172.21.15.96:0/1491974540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 msgr2=0x55a253d96c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 -- 172.21.15.96:0/1491974540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 msgr2=0x55a253d9bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253d9bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 -- 172.21.15.96:0/1491974540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a253d5e500 con 0x55a253d9c3b0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc78cb90640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253d9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc78d391640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc785ffb640 1 -- 172.21.15.96:0/3100384836 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc7740cebc0 con 0x7fc7881029e0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.226+0000 7fc787fff640 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253d9e840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc7740cc540 tx=0x7fc774066190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ae6671d6d9c326 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc78d391640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 0x7fc7780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc772ffd640 1 -- 172.21.15.96:0/1491974540 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77409ad30 con 0x55a253d9c3b0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc772ffd640 1 -- 172.21.15.96:0/1491974540 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409bad0 con 0x55a253d9c3b0 2024-03-18T15:40:45.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc772ffd640 1 -- 172.21.15.96:0/1491974540 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77409b3d0 con 0x55a253d9c3b0 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 msgr2=0x55a253d9e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253d9e840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc7740cc540 tx=0x7fc774066190 comp rx=0 tx=0).stop 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.227+0000 7fc78d391640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 0x7fc7780a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc780033a30 tx=0x7fc780009300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 shutdown_connections 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253d9e840 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253d9bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1491974540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253d96c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 >> 172.21.15.96:0/1491974540 conn(0x55a253d91dc0 msgr2=0x55a253d921c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 shutdown_connections 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- 172.21.15.96:0/1491974540 wait complete. 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- start start 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 Processor -- start 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 0x55a253da0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d9c3b0 0x55a253da8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a253d9f300 con 0x55a253d96850 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a253cb7230 con 0x55a253d9c3b0 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a253d9f480 con 0x55a253d97240 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.228+0000 7fc7919e4740 10 client.0 _ll_register_callbacks cb 0x55a253cf3010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39706/0 (socket says 172.21.15.96:39706) 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 -- 172.21.15.96:0/436022056 learned_addr learned my addr 172.21.15.96:0/436022056 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc78cb90640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 0x55a253da0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc78d391640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d9c3b0 0x55a253da8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 -- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d9c3b0 msgr2=0x55a253da8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d9c3b0 0x55a253da8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 -- 172.21.15.96:0/436022056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 msgr2=0x55a253da0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 0x55a253da0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 -- 172.21.15.96:0/436022056 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a253cc6100 con 0x55a253d97240 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc78cb90640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 0x55a253da0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc787fff640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c2ce0 tx=0x7fc7740d2780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728a3a47d7c96e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc767fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77401c040 con 0x55a253d97240 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77409bad0 con 0x55a253d97240 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc7740af470 con 0x55a253d97240 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.229+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc77409ad30 con 0x55a253d97240 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 10 client.25453 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a253a0f3f0 con 0x55a253d97240 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409bad0 con 0x55a253d97240 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc7740d2260 con 0x55a253d97240 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 10 client.25453 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 10 client.25453 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 20 client.25453 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253dac730 0x55a253daeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 4 client.25453 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a253d089d0 con 0x55a253dac730 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.230+0000 7fc78cb90640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253dac730 0x55a253daeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc767fff640 10 client.25453 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc78cb90640 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253dac730 0x55a253daeb40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4042f44ed6962e9 server_cookie=72436d19085311ad in_seq=0 out_seq=0 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc767fff640 1 -- 172.21.15.96:0/436022056 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a253d089d0 con 0x55a253dac730 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc767fff640 10 client.25453 handle_command_reply: tid=0 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc7919e4740 1 client.25453 shutdown 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 msgr2=0x55a253da1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.231+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc7740c2ce0 tx=0x7fc7740d2780 comp rx=0 tx=0).stop 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 shutdown_connections 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253dac730 0x55a253daeb40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d9c3b0 0x55a253da8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d97240 0x55a253da1210 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc767fff640 0 client.25453 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 --2- 172.21.15.96:0/436022056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d96850 0x55a253da0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 >> 172.21.15.96:0/436022056 conn(0x55a253d91dc0 msgr2=0x55a253d90fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 shutdown_connections 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 1 -- 172.21.15.96:0/436022056 wait complete. 2024-03-18T15:40:45.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.232+0000 7fc7919e4740 20 client.25453 trim_cache size 0 max 16384 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 Processor -- start 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 -- start start 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 0x55a253d9ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 0x55a253da8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d9c3b0 0x55a253db3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881a8cd0 con 0x55a253d97240 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881aa810 con 0x55a253d96850 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc7919e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881a93a0 con 0x55a253d9c3b0 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc787fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 0x55a253da8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc78cb90640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 0x55a253d9ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc787fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 0x55a253da8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54022/0 (socket says 172.21.15.96:54022) 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc787fff640 1 -- 172.21.15.96:0/708657170 learned_addr learned my addr 172.21.15.96:0/708657170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.245+0000 7fc78d391640 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d9c3b0 0x55a253db3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 -- 172.21.15.96:0/708657170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d9c3b0 msgr2=0x55a253db3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d9c3b0 0x55a253db3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 -- 172.21.15.96:0/708657170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 msgr2=0x55a253da8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 0x55a253da8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 -- 172.21.15.96:0/708657170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a253ad97d0 con 0x55a253d96850 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc78cb90640 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 0x55a253d9ff60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55a253daf380 tx=0x7fc77c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b23aba7993c7e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc7727fc640 1 -- 172.21.15.96:0/708657170 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004220 con 0x55a253d96850 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc7727fc640 1 -- 172.21.15.96:0/708657170 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77c0043c0 con 0x55a253d96850 2024-03-18T15:40:45.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.246+0000 7fc7727fc640 1 -- 172.21.15.96:0/708657170 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c020050 con 0x55a253d96850 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 -- 172.21.15.96:0/708657170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 msgr2=0x55a253d9ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 0x55a253d9ff60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55a253daf380 tx=0x7fc77c009260 comp rx=0 tx=0).stop 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 -- 172.21.15.96:0/708657170 shutdown_connections 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d9c3b0 0x55a253db3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d97240 0x55a253da8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 --2- 172.21.15.96:0/708657170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d96850 0x55a253d9ff60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 -- 172.21.15.96:0/708657170 >> 172.21.15.96:0/708657170 conn(0x55a253d91dc0 msgr2=0x55a253d9f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 -- 172.21.15.96:0/708657170 shutdown_connections 2024-03-18T15:40:45.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.247+0000 7fc7919e4740 1 -- 172.21.15.96:0/708657170 wait complete. 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 -- start start 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 Processor -- start 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253da4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253da6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881aa140 con 0x55a253d9c3b0 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881a9a70 con 0x55a253d97240 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc7881a6cb0 con 0x55a253d96850 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc78cb90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc78cb90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39730/0 (socket says 172.21.15.96:39730) 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc78cb90640 1 -- 172.21.15.96:0/1091061524 learned_addr learned my addr 172.21.15.96:0/1091061524 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc7919e4740 10 client.0 _ll_register_callbacks cb 0x7fc77c009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.248+0000 7fc787fff640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253da4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78d391640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253da6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 -- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 msgr2=0x55a253da4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253da4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 -- 172.21.15.96:0/1091061524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 msgr2=0x55a253da6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253da6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 -- 172.21.15.96:0/1091061524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a253da75b0 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78d391640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253da6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc78cb90640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc77c023040 tx=0x7fc77c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c138f53a000cf54 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc767fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c02e060 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc77c004f30 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77c0043c0 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc77c025040 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc7919e4740 10 client.25459 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.249+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a253e16fa0 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77c020160 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc77c029420 con 0x55a253d96850 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 10 client.25459 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 10 client.25459 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 20 client.25459 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253e17920 0x55a253e19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 4 client.25459 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc77c0043c0 con 0x55a253e17920 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.250+0000 7fc787fff640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253e17920 0x55a253e19d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc767fff640 10 client.25459 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc787fff640 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253e17920 0x55a253e19d40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=582fada9e3a99f7 server_cookie=4d74fed4c239bcf9 in_seq=0 out_seq=0 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc767fff640 1 -- 172.21.15.96:0/1091061524 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc77c0043c0 con 0x55a253e17920 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc767fff640 10 client.25459 handle_command_reply: tid=0 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 client.25459 shutdown 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 msgr2=0x55a253da3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc77c023040 tx=0x7fc77c004d00 comp rx=0 tx=0).stop 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 shutdown_connections 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55a253e17920 0x55a253e19d40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a253d9c3b0 0x55a253da6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc767fff640 0 client.25459 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a253d97240 0x55a253da4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 --2- 172.21.15.96:0/1091061524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a253d96850 0x55a253da3b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 >> 172.21.15.96:0/1091061524 conn(0x55a253d91dc0 msgr2=0x55a253da00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 shutdown_connections 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 1 -- 172.21.15.96:0/1091061524 wait complete. 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.251+0000 7fc7919e4740 20 client.25459 trim_cache size 0 max 16384 2024-03-18T15:40:45.257 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 msgr2=0x7fc7780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 0x7fc7780a4d50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc780033a30 tx=0x7fc780009300 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 msgr2=0x7fc788198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc77400e9f0 tx=0x7fc77400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 shutdown_connections 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc7780a2890 0x7fc7780a4d50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7881033a0 0x7fc78819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7881029e0 0x7fc788198000 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc77400e9f0 tx=0x7fc77400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 --2- 172.21.15.96:0/3100384836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7881017f0 0x7fc788197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 >> 172.21.15.96:0/3100384836 conn(0x7fc7880fcd60 msgr2=0x7fc788105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 shutdown_connections 2024-03-18T15:40:45.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.252+0000 7fc78edf4640 1 -- 172.21.15.96:0/3100384836 wait complete. 2024-03-18T15:40:45.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 -- 172.21.15.96:0/246819400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c1033c0 msgr2=0x7fd35c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 --2- 172.21.15.96:0/246819400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c1033c0 0x7fd35c109c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd354009be0 tx=0x7fd35402bef0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 -- 172.21.15.96:0/246819400 shutdown_connections 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 --2- 172.21.15.96:0/246819400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c1033c0 0x7fd35c109c50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 --2- 172.21.15.96:0/246819400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c102a00 0x7fd35c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 --2- 172.21.15.96:0/246819400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c101810 0x7fd35c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 -- 172.21.15.96:0/246819400 >> 172.21.15.96:0/246819400 conn(0x7fd35c0fcd80 msgr2=0x7fd35c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 -- 172.21.15.96:0/246819400 shutdown_connections 2024-03-18T15:40:45.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.374+0000 7fd361c6c640 1 -- 172.21.15.96:0/246819400 wait complete. 2024-03-18T15:40:45.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 Processor -- start 2024-03-18T15:40:45.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 -- start start 2024-03-18T15:40:45.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 0x7fd35c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 0x7fd35c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c075140 con 0x7fd35c1033c0 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c074fc0 con 0x7fd35c101810 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd361c6c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0752c0 con 0x7fd35c102a00 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd35bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 0x7fd35c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51342/0 (socket says 172.21.15.96:51342) 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3891658467 learned_addr learned my addr 172.21.15.96:0/3891658467 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.375+0000 7fd35affd640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 0x7fd35c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 -- 172.21.15.96:0/3891658467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 msgr2=0x7fd35c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 0x7fd35c19a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 -- 172.21.15.96:0/3891658467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 msgr2=0x7fd35c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 -- 172.21.15.96:0/3891658467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1a18d0 con 0x7fd35c1033c0 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35affd640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 0x7fd35c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 0x7fd35c1a1230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd35402c430 tx=0x7fd3540337b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd358ff9640 1 -- 172.21.15.96:0/3891658467 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354043020 con 0x7fd35c1033c0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd358ff9640 1 -- 172.21.15.96:0/3891658467 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35402ca60 con 0x7fd35c1033c0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd358ff9640 1 -- 172.21.15.96:0/3891658467 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354034690 con 0x7fd35c1033c0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1a1b60 con 0x7fd35c1033c0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.376+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1a2060 con 0x7fd35c1033c0 2024-03-18T15:40:45.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.378+0000 7fd358ff9640 1 -- 172.21.15.96:0/3891658467 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fd354046020 con 0x7fd35c1033c0 2024-03-18T15:40:45.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.379+0000 7fd358ff9640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 0x7fd3480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.379+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 0x7fd3480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.379+0000 7fd358ff9640 1 -- 172.21.15.96:0/3891658467 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fd3540f0fc0 con 0x7fd35c1033c0 2024-03-18T15:40:45.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 0x7fd3480a4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd35c19ab60 tx=0x7fd344009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 Processor -- start 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 -- start start 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f40240 0x55ef96f44d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f47840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef96ea4490 con 0x55ef96f40240 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef96e4cc90 con 0x55ef96f453b0 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd364858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef96ea0530 con 0x55ef96f3f850 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f40240 0x55ef96f44d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39754/0 (socket says 172.21.15.96:39754) 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3263153778 learned_addr learned my addr 172.21.15.96:0/3263153778 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f47840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3263153778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 msgr2=0x55ef96f47840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f47840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3263153778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f40240 msgr2=0x55ef96f44d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f40240 0x55ef96f44d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.380+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3263153778 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef96f07500 con 0x55ef96f3f850 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd344004340 tx=0x7fd344014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba285948755e803 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd351ffb640 1 -- 172.21.15.96:0/3263153778 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344027470 con 0x55ef96f3f850 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd351ffb640 1 -- 172.21.15.96:0/3263153778 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344022070 con 0x55ef96f3f850 2024-03-18T15:40:45.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd351ffb640 1 -- 172.21.15.96:0/3263153778 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34401d900 con 0x55ef96f3f850 2024-03-18T15:40:45.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 -- 172.21.15.96:0/3263153778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 msgr2=0x55ef96f3fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd344004340 tx=0x7fd344014000 comp rx=0 tx=0).stop 2024-03-18T15:40:45.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 -- 172.21.15.96:0/3263153778 shutdown_connections 2024-03-18T15:40:45.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f47840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f40240 0x55ef96f44d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 --2- 172.21.15.96:0/3263153778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f3fc70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.381+0000 7fd364858740 1 -- 172.21.15.96:0/3263153778 >> 172.21.15.96:0/3263153778 conn(0x55ef96f3adc0 msgr2=0x55ef96f3b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- 172.21.15.96:0/3263153778 shutdown_connections 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- 172.21.15.96:0/3263153778 wait complete. 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- start start 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 Processor -- start 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f40240 0x55ef96f4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef96f48360 con 0x55ef96f3f850 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ef96e60230 con 0x55ef96f453b0 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd364858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef96f484e0 con 0x55ef96f40240 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd35bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54062/0 (socket says 172.21.15.96:54062) 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd35bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51366/0 (socket says 172.21.15.96:51366) 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.382+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3142949899 learned_addr learned my addr 172.21.15.96:0/3142949899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35affd640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f40240 0x55ef96f4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd364858740 10 client.0 _ll_register_callbacks cb 0x55ef96e9c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f40240 msgr2=0x55ef96f4a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f40240 0x55ef96f4a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3142949899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 msgr2=0x55ef96f512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3142949899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef96e6f100 con 0x55ef96f3f850 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd344004310 tx=0x7fd34400f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc36b20704f4a1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd33affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34400fae0 con 0x55ef96f3f850 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fd344005050 con 0x55ef96f3f850 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd344022070 con 0x55ef96f3f850 2024-03-18T15:40:45.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd344005250 con 0x55ef96f3f850 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd364858740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.383+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef96bb83f0 con 0x55ef96f3f850 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344022070 con 0x55ef96f3f850 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd34401d640 con 0x55ef96f3f850 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 10 client.15624 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 20 client.15624 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96f57c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 4 client.15624 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef96eb19d0 con 0x55ef96f557b0 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd35affd640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96f57c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd35affd640 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96f57c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8434a7e22923807b server_cookie=53dc55127d3a6b77 in_seq=0 out_seq=0 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.384+0000 7fd33affd640 10 client.15624 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd33affd640 1 -- 172.21.15.96:0/3142949899 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ef96eb19d0 con 0x55ef96f557b0 2024-03-18T15:40:45.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd33affd640 10 client.15624 handle_command_reply: tid=0 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 client.15624 shutdown 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 msgr2=0x55ef96f49d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd344004310 tx=0x7fd34400f940 comp rx=0 tx=0).stop 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 shutdown_connections 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96f57c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd33affd640 0 client.15624 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f453b0 0x55ef96f512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f40240 0x55ef96f4a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 --2- 172.21.15.96:0/3142949899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f49d90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 >> 172.21.15.96:0/3142949899 conn(0x55ef96f3adc0 msgr2=0x55ef96f39fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 shutdown_connections 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.385+0000 7fd364858740 1 -- 172.21.15.96:0/3142949899 wait complete. 2024-03-18T15:40:45.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.386+0000 7fd364858740 20 client.15624 trim_cache size 0 max 16384 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 Processor -- start 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 -- start start 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f51260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 0x55ef96f49ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1ab000 con 0x55ef96f453b0 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1acb40 con 0x55ef96f40240 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd364858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1ab6d0 con 0x55ef96f3f850 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f51260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39774/0 (socket says 172.21.15.96:39774) 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f51260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51368/0 (socket says 172.21.15.96:51368) 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3886786875 learned_addr learned my addr 172.21.15.96:0/3886786875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 0x55ef96f49ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3886786875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 msgr2=0x55ef96f51260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f51260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3886786875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 msgr2=0x55ef96f49ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 0x55ef96f49ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35b7fe640 1 -- 172.21.15.96:0/3886786875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef96c827d0 con 0x55ef96f3f850 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.399+0000 7fd35bfff640 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 0x55ef96f49ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.400+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd34401da90 tx=0x7fd34400ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87c5e403831aeb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.400+0000 7fd3517fa640 1 -- 172.21.15.96:0/3886786875 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344003f80 con 0x55ef96f3f850 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.400+0000 7fd3517fa640 1 -- 172.21.15.96:0/3886786875 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344022040 con 0x55ef96f3f850 2024-03-18T15:40:45.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.400+0000 7fd3517fa640 1 -- 172.21.15.96:0/3886786875 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344027470 con 0x55ef96f3f850 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- 172.21.15.96:0/3886786875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 msgr2=0x55ef96f499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd34401da90 tx=0x7fd34400ffd0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- 172.21.15.96:0/3886786875 shutdown_connections 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f453b0 0x55ef96f49ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f51260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 --2- 172.21.15.96:0/3886786875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f3f850 0x55ef96f499a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- 172.21.15.96:0/3886786875 >> 172.21.15.96:0/3886786875 conn(0x55ef96f3adc0 msgr2=0x55ef96f41700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- 172.21.15.96:0/3886786875 shutdown_connections 2024-03-18T15:40:45.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- 172.21.15.96:0/3886786875 wait complete. 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 -- start start 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.401+0000 7fd364858740 1 Processor -- start 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f4e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 0x55ef96fb9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1ac470 con 0x55ef96f3f850 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1abda0 con 0x55ef96f40240 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c1a8fe0 con 0x55ef96f453b0 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd364858740 10 client.0 _ll_register_callbacks cb 0x7fd344004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54076/0 (socket says 172.21.15.96:54076) 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35b7fe640 1 -- 172.21.15.96:0/2747388561 learned_addr learned my addr 172.21.15.96:0/2747388561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35affd640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f4e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 0x55ef96fb9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 -- 172.21.15.96:0/2747388561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 msgr2=0x55ef96f4e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f4e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 -- 172.21.15.96:0/2747388561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 msgr2=0x55ef96f4dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 -- 172.21.15.96:0/2747388561 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef96f51410 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd35bfff640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 0x55ef96fb9af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3540eea50 tx=0x7fd3540b9c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14f08d6fa5e947dc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.402+0000 7fd33affd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3540d6030 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fd354004b40 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd344022040 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd3540bdfe0 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd364858740 10 client.25477 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef96fbb000 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd344022040 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.403+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd3540bcd30 con 0x55ef96f453b0 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 10 client.25477 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 10 client.25477 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 20 client.25477 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96fbda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 4 client.25477 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef96eabd20 con 0x55ef96f557b0 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96fbda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd33affd640 10 client.25477 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd35b7fe640 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96fbda10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ead8692ca063ef64 server_cookie=5ff2fa6d1263d52c in_seq=0 out_seq=0 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd33affd640 1 -- 172.21.15.96:0/2747388561 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef96eabd20 con 0x55ef96f557b0 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd33affd640 10 client.25477 handle_command_reply: tid=0 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.404+0000 7fd364858740 1 client.25477 shutdown 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 msgr2=0x55ef96fb9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 0x55ef96fb9af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd3540eea50 tx=0x7fd3540b9c40 comp rx=0 tx=0).stop 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 shutdown_connections 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55ef96f557b0 0x55ef96fbda10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd33affd640 0 client.25477 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef96f453b0 0x55ef96fb9af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ef96f40240 0x55ef96f4e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 --2- 172.21.15.96:0/2747388561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef96f3f850 0x55ef96f4dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 >> 172.21.15.96:0/2747388561 conn(0x55ef96f3adc0 msgr2=0x55ef96f49910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 shutdown_connections 2024-03-18T15:40:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 1 -- 172.21.15.96:0/2747388561 wait complete. 2024-03-18T15:40:45.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd364858740 20 client.25477 trim_cache size 0 max 16384 2024-03-18T15:40:45.411 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 msgr2=0x7fd3480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 0x7fd3480a4dd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd35c19ab60 tx=0x7fd344009260 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 msgr2=0x7fd35c1a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 0x7fd35c1a1230 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd35402c430 tx=0x7fd3540337b0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 shutdown_connections 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd3480a2910 0x7fd3480a4dd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35c1033c0 0x7fd35c1a1230 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c102a00 0x7fd35c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 --2- 172.21.15.96:0/3891658467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd35c101810 0x7fd35c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.405+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 >> 172.21.15.96:0/3891658467 conn(0x7fd35c0fcd80 msgr2=0x7fd35c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.406+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 shutdown_connections 2024-03-18T15:40:45.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.406+0000 7fd361c6c640 1 -- 172.21.15.96:0/3891658467 wait complete. 2024-03-18T15:40:45.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:40:45.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 -- 172.21.15.96:0/2681171410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 msgr2=0x7fb980101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 --2- 172.21.15.96:0/2681171410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980101d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb974009900 tx=0x7fb97402bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 -- 172.21.15.96:0/2681171410 shutdown_connections 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 --2- 172.21.15.96:0/2681171410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb980108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 --2- 172.21.15.96:0/2681171410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980101d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 --2- 172.21.15.96:0/2681171410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb980100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 -- 172.21.15.96:0/2681171410 >> 172.21.15.96:0/2681171410 conn(0x7fb9800fbc70 msgr2=0x7fb9800fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 -- 172.21.15.96:0/2681171410 shutdown_connections 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.521+0000 7fb9866dc640 1 -- 172.21.15.96:0/2681171410 wait complete. 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 Processor -- start 2024-03-18T15:40:45.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 -- start start 2024-03-18T15:40:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb98019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010b260 con 0x7fb9801006e0 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010b0e0 con 0x7fb980102290 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.522+0000 7fb9866dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb98010b3e0 con 0x7fb9801018d0 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54104/0 (socket says 172.21.15.96:54104) 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 -- 172.21.15.96:0/98564489 learned_addr learned my addr 172.21.15.96:0/98564489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb984c79640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb98019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 -- 172.21.15.96:0/98564489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 msgr2=0x7fb980195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 -- 172.21.15.96:0/98564489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 msgr2=0x7fb98019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb98019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 -- 172.21.15.96:0/98564489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98019d4e0 con 0x7fb9801006e0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb984c79640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb98019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97ffff640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9c0 tx=0x7fb97000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97d7fa640 1 -- 172.21.15.96:0/98564489 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000cd80 con 0x7fb9801006e0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97d7fa640 1 -- 172.21.15.96:0/98564489 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700035f0 con 0x7fb9801006e0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98019d7d0 con 0x7fb9801006e0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.523+0000 7fb97d7fa640 1 -- 172.21.15.96:0/98564489 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970010690 con 0x7fb9801006e0 2024-03-18T15:40:45.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.524+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98019dd20 con 0x7fb9801006e0 2024-03-18T15:40:45.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 Processor -- start 2024-03-18T15:40:45.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 -- start start 2024-03-18T15:40:45.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c3240 0x561afe8c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 0x561afe8ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561afe827490 con 0x561afe8c2850 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561afe7cfc90 con 0x561afe8c83b0 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.526+0000 7fb9893a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561afe823530 con 0x561afe8c3240 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54112/0 (socket says 172.21.15.96:54112) 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 -- 172.21.15.96:0/1607486356 learned_addr learned my addr 172.21.15.96:0/1607486356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 0x561afe8ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb97d7fa640 1 -- 172.21.15.96:0/98564489 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fb9700037a0 con 0x7fb9801006e0 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 -- 172.21.15.96:0/1607486356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c3240 msgr2=0x561afe8c7d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c3240 0x561afe8c7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 -- 172.21.15.96:0/1607486356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 msgr2=0x561afe8ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 0x561afe8ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 -- 172.21.15.96:0/1607486356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afe88a500 con 0x561afe8c2850 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 0x561afe8ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb984c79640 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb978003ae0 tx=0x7fb97800b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9c3495681066d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb9667fc640 1 -- 172.21.15.96:0/1607486356 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97801a030 con 0x561afe8c2850 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb9667fc640 1 -- 172.21.15.96:0/1607486356 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9780046c0 con 0x561afe8c2850 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb9667fc640 1 -- 172.21.15.96:0/1607486356 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780168f0 con 0x561afe8c2850 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.527+0000 7fb97d7fa640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 0x7fb9680a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb97d7fa640 1 -- 172.21.15.96:0/98564489 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb9700cf020 con 0x7fb9801006e0 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb97ffff640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 0x7fb9680a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 -- 172.21.15.96:0/1607486356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 msgr2=0x561afe8c2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb97ffff640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 0x7fb9680a4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x561afe8c3cb0 tx=0x7fb9700cf2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb978003ae0 tx=0x7fb97800b710 comp rx=0 tx=0).stop 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 -- 172.21.15.96:0/1607486356 shutdown_connections 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c83b0 0x561afe8ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c3240 0x561afe8c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1607486356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c2850 0x561afe8c2c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 -- 172.21.15.96:0/1607486356 >> 172.21.15.96:0/1607486356 conn(0x561afe8bddc0 msgr2=0x561afe8be1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 -- 172.21.15.96:0/1607486356 shutdown_connections 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.528+0000 7fb9893a5740 1 -- 172.21.15.96:0/1607486356 wait complete. 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 -- start start 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 Processor -- start 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561afe8cf770 con 0x561afe8c83b0 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561afe7e3230 con 0x561afe8c3240 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561afe8cf8f0 con 0x561afe8c2850 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.529+0000 7fb9893a5740 10 client.0 _ll_register_callbacks cb 0x561afe81f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb97f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54122/0 (socket says 172.21.15.96:54122) 2024-03-18T15:40:45.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 -- 172.21.15.96:0/1321604509 learned_addr learned my addr 172.21.15.96:0/1321604509 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 -- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 msgr2=0x561afe8d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 -- 172.21.15.96:0/1321604509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 msgr2=0x561afe8d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 -- 172.21.15.96:0/1321604509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afe7f2100 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb984c79640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb978010de0 tx=0x7fb978010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11479be7237a9926 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978017910 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb978017ab0 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9780046c0 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb978017cb0 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.530+0000 7fb9893a5740 10 client.15642 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561afe53b3f0 con 0x561afe8c83b0 2024-03-18T15:40:45.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9780046c0 con 0x561afe8c83b0 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb978037070 con 0x561afe8c83b0 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 10 client.15642 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 10 client.15642 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 20 client.15642 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe8df2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 4 client.15642 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561afe8349d0 con 0x561afe8dcee0 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.531+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe8df2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe8df2d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df9ebff4e02151b0 server_cookie=74083270bc3026ee in_seq=0 out_seq=0 2024-03-18T15:40:45.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb95f7fe640 10 client.15642 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1321604509 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561afe8349d0 con 0x561afe8dcee0 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb95f7fe640 10 client.15642 handle_command_reply: tid=0 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb9893a5740 1 client.15642 shutdown 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 msgr2=0x561afe8d86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb978010de0 tx=0x7fb978010d30 comp rx=0 tx=0).stop 2024-03-18T15:40:45.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 shutdown_connections 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.532+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe8df2d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe8d86b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb95f7fe640 0 client.15642 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1321604509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 >> 172.21.15.96:0/1321604509 conn(0x561afe8bddc0 msgr2=0x561afe8bcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 shutdown_connections 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 1 -- 172.21.15.96:0/1321604509 wait complete. 2024-03-18T15:40:45.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.533+0000 7fb9893a5740 20 client.15642 trim_cache size 0 max 16384 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 Processor -- start 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 -- start start 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 0x561afe8c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c83b0 0x561afe8d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a6c90 con 0x561afe8c3240 2024-03-18T15:40:45.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a87d0 con 0x561afe8c2850 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.556+0000 7fb9893a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a7360 con 0x561afe8c83b0 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51404/0 (socket says 172.21.15.96:51404) 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 -- 172.21.15.96:0/555938806 learned_addr learned my addr 172.21.15.96:0/555938806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c83b0 0x561afe8d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 0x561afe8c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 -- 172.21.15.96:0/555938806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c83b0 msgr2=0x561afe8d8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c83b0 0x561afe8d8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 -- 172.21.15.96:0/555938806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 msgr2=0x561afe8c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 0x561afe8c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 -- 172.21.15.96:0/555938806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561afe8df640 con 0x561afe8c2850 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 0x561afe8c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb984c79640 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb978011010 tx=0x7fb978005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8d8bf21d8797738 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.557+0000 7fb965ffb640 1 -- 172.21.15.96:0/555938806 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97801a040 con 0x561afe8c2850 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.558+0000 7fb965ffb640 1 -- 172.21.15.96:0/555938806 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb978034070 con 0x561afe8c2850 2024-03-18T15:40:45.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.558+0000 7fb965ffb640 1 -- 172.21.15.96:0/555938806 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978017910 con 0x561afe8c2850 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.558+0000 7fb9893a5740 1 -- 172.21.15.96:0/555938806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 msgr2=0x561afe8c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.558+0000 7fb9893a5740 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb978011010 tx=0x7fb978005fd0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 -- 172.21.15.96:0/555938806 shutdown_connections 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c83b0 0x561afe8d8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c3240 0x561afe8c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 --2- 172.21.15.96:0/555938806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c2850 0x561afe8c4780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 -- 172.21.15.96:0/555938806 >> 172.21.15.96:0/555938806 conn(0x561afe8bddc0 msgr2=0x561afe8be1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 -- 172.21.15.96:0/555938806 shutdown_connections 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 -- 172.21.15.96:0/555938806 wait complete. 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 -- start start 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.559+0000 7fb9893a5740 1 Processor -- start 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe9411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a8100 con 0x561afe8c83b0 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a7a30 con 0x561afe8c3240 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb9893a5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801a4c70 con 0x561afe8c2850 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39828/0 (socket says 172.21.15.96:39828) 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 -- 172.21.15.96:0/1874209375 learned_addr learned my addr 172.21.15.96:0/1874209375 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe9411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 -- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 msgr2=0x561afe8d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 -- 172.21.15.96:0/1874209375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 msgr2=0x561afe8d5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb984c79640 1 -- 172.21.15.96:0/1874209375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561afe6057d0 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.560+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb984c79640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe9411a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb978010de0 tx=0x7fb9780047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff86b7244cac764b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1874209375 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97803f040 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb9893a5740 10 client.0 _ll_register_callbacks cb 0x7fb978003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1874209375 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb978030030 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb978034070 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1874209375 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb97804b030 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb9893a5740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561afe942280 con 0x561afe8c83b0 2024-03-18T15:40:45.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.561+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1874209375 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb978037070 con 0x561afe8c83b0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 10 client.15648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 20 client.15648 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe944c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 4 client.15648 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb978034070 con 0x561afe8dcee0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe944c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb95f7fe640 10 client.15648 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb97f7fe640 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe944c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=377cabcbd5f3c92f server_cookie=151584261ad7fc43 in_seq=0 out_seq=0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb95f7fe640 1 -- 172.21.15.96:0/1874209375 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb978034070 con 0x561afe8dcee0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.562+0000 7fb95f7fe640 10 client.15648 handle_command_reply: tid=0 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 client.15648 shutdown 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 msgr2=0x561afe9411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe9411a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb978010de0 tx=0x7fb9780047d0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 shutdown_connections 2024-03-18T15:40:45.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561afe8dcee0 0x561afe944c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561afe8c83b0 0x561afe9411a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb95f7fe640 0 client.15648 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561afe8c3240 0x561afe8d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 --2- 172.21.15.96:0/1874209375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561afe8c2850 0x561afe8d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 >> 172.21.15.96:0/1874209375 conn(0x561afe8bddc0 msgr2=0x561afe8cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 shutdown_connections 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 1 -- 172.21.15.96:0/1874209375 wait complete. 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.563+0000 7fb9893a5740 20 client.15648 trim_cache size 0 max 16384 2024-03-18T15:40:45.568 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:40:45.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 msgr2=0x7fb9680a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 0x7fb9680a4d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x561afe8c3cb0 tx=0x7fb9700cf2e0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 msgr2=0x7fb9801958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9c0 tx=0x7fb97000ee80 comp rx=0 tx=0).stop 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 shutdown_connections 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb9680a2890 0x7fb9680a4d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb980102290 0x7fb98019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9801018d0 0x7fb980195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 --2- 172.21.15.96:0/98564489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9801006e0 0x7fb9801958c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 >> 172.21.15.96:0/98564489 conn(0x7fb9800fbc70 msgr2=0x7fb980104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 shutdown_connections 2024-03-18T15:40:45.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.564+0000 7fb9866dc640 1 -- 172.21.15.96:0/98564489 wait complete. 2024-03-18T15:40:45.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:40:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 -- 172.21.15.96:0/208469272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901006e0 msgr2=0x7f3d90100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 --2- 172.21.15.96:0/208469272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901006e0 0x7f3d90100b00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009900 tx=0x7f3d8002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 -- 172.21.15.96:0/208469272 shutdown_connections 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 --2- 172.21.15.96:0/208469272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d90102290 0x7f3d90108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 --2- 172.21.15.96:0/208469272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901018d0 0x7f3d90101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 --2- 172.21.15.96:0/208469272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901006e0 0x7f3d90100b00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 -- 172.21.15.96:0/208469272 >> 172.21.15.96:0/208469272 conn(0x7f3d900fbc70 msgr2=0x7f3d900fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 -- 172.21.15.96:0/208469272 shutdown_connections 2024-03-18T15:40:45.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 -- 172.21.15.96:0/208469272 wait complete. 2024-03-18T15:40:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.684+0000 7f3d95212640 1 Processor -- start 2024-03-18T15:40:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 -- start start 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 0x7f3d90197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d90102290 0x7f3d9019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010d440 con 0x7f3d90102290 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010d2c0 con 0x7f3d901018d0 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d95212640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010d5c0 con 0x7f3d901006e0 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 0x7f3d90197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 0x7f3d90197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39836/0 (socket says 172.21.15.96:39836) 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/1892514628 learned_addr learned my addr 172.21.15.96:0/1892514628 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51416/0 (socket says 172.21.15.96:51416) 2024-03-18T15:40:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d90102290 0x7f3d9019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:45 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4257473599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 -- 172.21.15.96:0/1892514628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 msgr2=0x7f3d90197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 0x7f3d90197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 -- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d90102290 msgr2=0x7f3d9019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d90102290 0x7f3d9019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.685+0000 7f3d8e59c640 1 -- 172.21.15.96:0/1892514628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9019f7a0 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400e9f0 tx=0x7f3d8400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d77fff640 1 -- 172.21.15.96:0/1892514628 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400ccc0 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d77fff640 1 -- 172.21.15.96:0/1892514628 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d840035f0 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d77fff640 1 -- 172.21.15.96:0/1892514628 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d8400ce60 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d90109540 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.686+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d90109a90 con 0x7f3d901018d0 2024-03-18T15:40:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.688+0000 7f3d77fff640 1 -- 172.21.15.96:0/1892514628 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3d840037a0 con 0x7f3d901018d0 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.688+0000 7f3d97e00740 1 Processor -- start 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 -- start start 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bb98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bb9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56168baf8490 con 0x56168bb94240 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56168baa0c90 con 0x56168bb93850 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d97e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56168baf4530 con 0x56168bb993b0 2024-03-18T15:40:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d77fff640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 0x7f3d780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51418/0 (socket says 172.21.15.96:51418) 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bb98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 -- 172.21.15.96:0/3466760893 learned_addr learned my addr 172.21.15.96:0/3466760893 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d77fff640 1 -- 172.21.15.96:0/1892514628 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3d840ceea0 con 0x7f3d901018d0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 0x7f3d780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bb9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 -- 172.21.15.96:0/3466760893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 msgr2=0x56168bb9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bb9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 -- 172.21.15.96:0/3466760893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 msgr2=0x56168bb98d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bb98d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8f59e640 1 -- 172.21.15.96:0/3466760893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168bb5b500 con 0x56168bb93850 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.689+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bb98d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.690+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 0x7f3d780a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800fc90 tx=0x7f3d8801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.690+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d88011e20 tx=0x7f3d8800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bf00fd3eb7e6d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.690+0000 7f3d757fa640 1 -- 172.21.15.96:0/3466760893 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d880115a0 con 0x56168bb93850 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.690+0000 7f3d757fa640 1 -- 172.21.15.96:0/3466760893 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d88003e50 con 0x56168bb93850 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.690+0000 7f3d757fa640 1 -- 172.21.15.96:0/3466760893 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004290 con 0x56168bb93850 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- 172.21.15.96:0/3466760893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 msgr2=0x56168bb93c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d88011e20 tx=0x7f3d8800f790 comp rx=0 tx=0).stop 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- 172.21.15.96:0/3466760893 shutdown_connections 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bb9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bb98d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3466760893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bb93c70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- 172.21.15.96:0/3466760893 >> 172.21.15.96:0/3466760893 conn(0x56168bb8edc0 msgr2=0x56168bb8f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- 172.21.15.96:0/3466760893 shutdown_connections 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- 172.21.15.96:0/3466760893 wait complete. 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- start start 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 Processor -- start 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb993b0 0x56168bbadad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56168bba4b10 con 0x56168bb993b0 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56168bab4230 con 0x56168bb93850 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56168bba4c90 con 0x56168bb94240 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.691+0000 7f3d97e00740 10 client.0 _ll_register_callbacks cb 0x56168baf0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51432/0 (socket says 172.21.15.96:51432) 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/990295436 learned_addr learned my addr 172.21.15.96:0/990295436 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39852/0 (socket says 172.21.15.96:39852) 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb993b0 0x56168bbadad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/990295436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 msgr2=0x56168bba6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/990295436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb993b0 msgr2=0x56168bbadad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb993b0 0x56168bbadad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/990295436 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168bac3100 con 0x56168bb93850 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d8002cfd0 tx=0x7f3d80009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71dc5e1aae0a25a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d800046c0 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3d80004860 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d88003e50 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d80004a60 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d97e00740 10 client.25451 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.692+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56168b80c3f0 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d88003e50 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d80003710 con 0x56168bb93850 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 10 client.25451 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 10 client.25451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 20 client.25451 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bbb22a0 0x56168bbb4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 4 client.25451 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56168bb059d0 con 0x56168bbb22a0 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bbb22a0 0x56168bbb4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d72ff5640 10 client.25451 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:40:45.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.693+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bbb22a0 0x56168bbb4740 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f36adb1e0b09cb6 server_cookie=9f01f8db78ecb098 in_seq=0 out_seq=0 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d72ff5640 1 -- 172.21.15.96:0/990295436 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56168bb059d0 con 0x56168bbb22a0 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d72ff5640 10 client.25451 handle_command_reply: tid=0 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 client.25451 shutdown 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 msgr2=0x56168bba6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d8002cfd0 tx=0x7f3d80009680 comp rx=0 tx=0).stop 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 shutdown_connections 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb993b0 0x56168bbadad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bba6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bbb22a0 0x56168bbb4740 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 --2- 172.21.15.96:0/990295436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba6550 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 >> 172.21.15.96:0/990295436 conn(0x56168bb8edc0 msgr2=0x56168bb8dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d72ff5640 0 client.25451 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 shutdown_connections 2024-03-18T15:40:45.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 1 -- 172.21.15.96:0/990295436 wait complete. 2024-03-18T15:40:45.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.694+0000 7f3d97e00740 20 client.25451 trim_cache size 0 max 16384 2024-03-18T15:40:45.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.707+0000 7f3d97e00740 1 Processor -- start 2024-03-18T15:40:45.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 -- start start 2024-03-18T15:40:45.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bba60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a6c80 con 0x56168bb94240 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a87c0 con 0x56168bb93850 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d97e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a7350 con 0x56168bb993b0 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54186/0 (socket says 172.21.15.96:54186) 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 -- 172.21.15.96:0/3646027068 learned_addr learned my addr 172.21.15.96:0/3646027068 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bba60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 -- 172.21.15.96:0/3646027068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 msgr2=0x56168bba60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bba60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 -- 172.21.15.96:0/3646027068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 msgr2=0x56168bba5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 -- 172.21.15.96:0/3646027068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168b8d67d0 con 0x56168bb94240 2024-03-18T15:40:45.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bba60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:45.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d84097a60 tx=0x7f3d8409b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e257b03c4c8a9ac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.708+0000 7f3d74ff9640 1 -- 172.21.15.96:0/3646027068 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840b9070 con 0x56168bb94240 2024-03-18T15:40:45.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d74ff9640 1 -- 172.21.15.96:0/3646027068 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d8409b650 con 0x56168bb94240 2024-03-18T15:40:45.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d74ff9640 1 -- 172.21.15.96:0/3646027068 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840b3070 con 0x56168bb94240 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 -- 172.21.15.96:0/3646027068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 msgr2=0x56168bbada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3d84097a60 tx=0x7f3d8409b550 comp rx=0 tx=0).stop 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 -- 172.21.15.96:0/3646027068 shutdown_connections 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb993b0 0x56168bba60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb94240 0x56168bbada90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 --2- 172.21.15.96:0/3646027068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb93850 0x56168bba5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 -- 172.21.15.96:0/3646027068 >> 172.21.15.96:0/3646027068 conn(0x56168bb8edc0 msgr2=0x56168bb928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 -- 172.21.15.96:0/3646027068 shutdown_connections 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.709+0000 7f3d97e00740 1 -- 172.21.15.96:0/3646027068 wait complete. 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 -- start start 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 Processor -- start 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bbab020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb993b0 0x56168bbac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a80f0 con 0x56168bb93850 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a7a20 con 0x56168bb993b0 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3d901a4c60 con 0x56168bb94240 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d97e00740 10 client.0 _ll_register_callbacks cb 0x7f3d80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bbab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54200/0 (socket says 172.21.15.96:54200) 2024-03-18T15:40:45.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/51701642 learned_addr learned my addr 172.21.15.96:0/51701642 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8f59e640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb993b0 0x56168bbac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/51701642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 msgr2=0x56168bbab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bbab020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb993b0 msgr2=0x56168bbac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.710+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb993b0 0x56168bbac030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d8ed9d640 1 -- 172.21.15.96:0/51701642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168bbac620 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bbab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d8ed9d640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3d80006010 tx=0x7f3d80034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d44c957a8eca84d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d800046c0 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3d80004860 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d8409b650 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d8005f030 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d97e00740 10 client.15666 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56168bbad920 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d8409b650 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.711+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d800035f0 con 0x56168bb93850 2024-03-18T15:40:45.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 10 client.15666 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 10 client.15666 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 20 client.15666 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bc189b0 0x56168bc1add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 4 client.15666 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56168baffd20 con 0x56168bc189b0 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bc189b0 0x56168bc1add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d72ff5640 10 client.15666 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d8e59c640 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bc189b0 0x56168bc1add0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d3a77f43e4ae4f0 server_cookie=db706f32776a6dad in_seq=0 out_seq=0 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d72ff5640 1 -- 172.21.15.96:0/51701642 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56168baffd20 con 0x56168bc189b0 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d72ff5640 10 client.15666 handle_command_reply: tid=0 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 1 client.15666 shutdown 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.712+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 msgr2=0x56168bbaaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3d80006010 tx=0x7f3d80034440 comp rx=0 tx=0).stop 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 shutdown_connections 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56168bc189b0 0x56168bc1add0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d72ff5640 0 client.15666 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56168bb993b0 0x56168bbac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56168bb94240 0x56168bbab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 --2- 172.21.15.96:0/51701642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56168bb93850 0x56168bbaaae0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 >> 172.21.15.96:0/51701642 conn(0x56168bb8edc0 msgr2=0x56168bb8d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 shutdown_connections 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 1 -- 172.21.15.96:0/51701642 wait complete. 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d97e00740 20 client.15666 trim_cache size 0 max 16384 2024-03-18T15:40:45.719 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.713+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 msgr2=0x7f3d780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 0x7f3d780a4d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800fc90 tx=0x7f3d8801e000 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 msgr2=0x7f3d901980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400e9f0 tx=0x7f3d8400edf0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 shutdown_connections 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3d780a2890 0x7f3d780a4d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d90102290 0x7f3d9019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3d901018d0 0x7f3d901980c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 --2- 172.21.15.96:0/1892514628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d901006e0 0x7f3d90197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 >> 172.21.15.96:0/1892514628 conn(0x7f3d900fbc70 msgr2=0x7f3d90104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 shutdown_connections 2024-03-18T15:40:45.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.714+0000 7f3d95212640 1 -- 172.21.15.96:0/1892514628 wait complete. 2024-03-18T15:40:45.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 -- 172.21.15.96:0/3500140974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c101810 msgr2=0x7f8f0c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/3500140974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c101810 0x7f8f0c101c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009900 tx=0x7f8f0802bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 -- 172.21.15.96:0/3500140974 shutdown_connections 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/3500140974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c1033c0 0x7f8f0c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/3500140974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/3500140974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c101810 0x7f8f0c101c30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 -- 172.21.15.96:0/3500140974 >> 172.21.15.96:0/3500140974 conn(0x7f8f0c0fcd80 msgr2=0x7f8f0c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 -- 172.21.15.96:0/3500140974 shutdown_connections 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.835+0000 7f8f13bbc640 1 -- 172.21.15.96:0/3500140974 wait complete. 2024-03-18T15:40:45.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 Processor -- start 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 -- start start 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c101810 0x7f8f0c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c1033c0 0x7f8f0c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10d490 con 0x7f8f0c102a00 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10d310 con 0x7f8f0c1033c0 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f13bbc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c10d610 con 0x7f8f0c101810 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54228/0 (socket says 172.21.15.96:54228) 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11157640 1 -- 172.21.15.96:0/2882921298 learned_addr learned my addr 172.21.15.96:0/2882921298 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11958640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c101810 0x7f8f0c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f12159640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c1033c0 0x7f8f0c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11157640 1 -- 172.21.15.96:0/2882921298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c101810 msgr2=0x7f8f0c197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.836+0000 7f8f11157640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c101810 0x7f8f0c197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f11157640 1 -- 172.21.15.96:0/2882921298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c1033c0 msgr2=0x7f8f0c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f11157640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c1033c0 0x7f8f0c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f11157640 1 -- 172.21.15.96:0/2882921298 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0c073a10 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f11157640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b2f0 tx=0x7f8efc00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f02ffd640 1 -- 172.21.15.96:0/2882921298 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc022070 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0c073d00 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f02ffd640 1 -- 172.21.15.96:0/2882921298 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8efc01ba00 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f02ffd640 1 -- 172.21.15.96:0/2882921298 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc01dba0 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.837+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0c0742a0 con 0x7f8f0c102a00 2024-03-18T15:40:45.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.839+0000 7f8f02ffd640 1 -- 172.21.15.96:0/2882921298 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f8efc025070 con 0x7f8f0c102a00 2024-03-18T15:40:45.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.840+0000 7f8f02ffd640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 0x7f8ef80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.840+0000 7f8f11958640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 0x7f8ef80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.840+0000 7f8f02ffd640 1 -- 172.21.15.96:0/2882921298 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8efc0d6b30 con 0x7f8f0c102a00 2024-03-18T15:40:45.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 Processor -- start 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 -- start start 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f11958640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 0x7f8ef80a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f08000c00 tx=0x7f8f08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 0x557ee767cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee767f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ee75dc490 con 0x557ee7677850 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557ee7584c90 con 0x557ee7678240 2024-03-18T15:40:45.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f1679c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ee75d8530 con 0x557ee767d3b0 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f11958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f11958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54234/0 (socket says 172.21.15.96:54234) 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f11157640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 0x557ee767cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.841+0000 7f8f11958640 1 -- 172.21.15.96:0/2400178416 learned_addr learned my addr 172.21.15.96:0/2400178416 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f12159640 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee767f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 -- 172.21.15.96:0/2400178416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 msgr2=0x557ee767f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee767f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 -- 172.21.15.96:0/2400178416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 msgr2=0x557ee767cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 0x557ee767cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 -- 172.21.15.96:0/2400178416 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee763f500 con 0x557ee7677850 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11157640 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 0x557ee767cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8f11958640 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005ec0 tx=0x7f8f0803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d43127447d552d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8ef7fff640 1 -- 172.21.15.96:0/2400178416 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0804c030 con 0x557ee7677850 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8ef7fff640 1 -- 172.21.15.96:0/2400178416 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08005530 con 0x557ee7677850 2024-03-18T15:40:45.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.842+0000 7f8ef7fff640 1 -- 172.21.15.96:0/2400178416 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08004da0 con 0x557ee7677850 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 -- 172.21.15.96:0/2400178416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 msgr2=0x557ee7677c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f08005ec0 tx=0x7f8f0803e000 comp rx=0 tx=0).stop 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 -- 172.21.15.96:0/2400178416 shutdown_connections 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee767f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7678240 0x557ee767cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2400178416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7677c70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 -- 172.21.15.96:0/2400178416 >> 172.21.15.96:0/2400178416 conn(0x557ee7672dc0 msgr2=0x557ee76731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 -- 172.21.15.96:0/2400178416 shutdown_connections 2024-03-18T15:40:45.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.843+0000 7f8f1679c740 1 -- 172.21.15.96:0/2400178416 wait complete. 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.844+0000 7f8f1679c740 1 -- start start 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.844+0000 7f8f1679c740 1 Processor -- start 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.844+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee7678240 0x557ee7682200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee767d3b0 0x557ee7689230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ee7680300 con 0x557ee7677850 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557ee7598230 con 0x557ee767d3b0 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54236/0 (socket says 172.21.15.96:54236) 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 -- 172.21.15.96:0/2175754556 learned_addr learned my addr 172.21.15.96:0/2175754556 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557ee7680480 con 0x557ee7678240 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f12159640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee767d3b0 0x557ee7689230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11157640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee7678240 0x557ee7682200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f1679c740 10 client.0 _ll_register_callbacks cb 0x557ee75d4010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 -- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee7678240 msgr2=0x557ee7682200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee7678240 0x557ee7682200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 -- 172.21.15.96:0/2175754556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee767d3b0 msgr2=0x557ee7689230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee767d3b0 0x557ee7689230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 -- 172.21.15.96:0/2175754556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee75a7100 con 0x557ee7677850 2024-03-18T15:40:45.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8f11958640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802c170 tx=0x7f8f08004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d9c41cde8941 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.845+0000 7f8ef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0804c040 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8f08005050 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f08005530 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f08005250 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8f1679c740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ee72f03f0 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.846+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08005530 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8f08048630 con 0x557ee7677850 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8f1679c740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8f1679c740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8f1679c740 20 client.15678 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee768d700 0x557ee768faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.847+0000 7f8f1679c740 4 client.15678 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.848+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ee75e99d0 con 0x557ee768d700 2024-03-18T15:40:45.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.848+0000 7f8f11157640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee768d700 0x557ee768faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.848+0000 7f8f11157640 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee768d700 0x557ee768faf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555b364e5928fac server_cookie=febdc4443e4da0bd in_seq=0 out_seq=0 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.848+0000 7f8ef57fa640 10 client.15678 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8ef57fa640 1 -- 172.21.15.96:0/2175754556 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557ee75e99d0 con 0x557ee768d700 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8ef57fa640 10 client.15678 handle_command_reply: tid=0 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 client.15678 shutdown 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 msgr2=0x557ee7681cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802c170 tx=0x7f8f08004b60 comp rx=0 tx=0).stop 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 shutdown_connections 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee767d3b0 0x557ee7689230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee7678240 0x557ee7682200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee768d700 0x557ee768faf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 --2- 172.21.15.96:0/2175754556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7677850 0x557ee7681cc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8ef57fa640 0 client.15678 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:45.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 >> 172.21.15.96:0/2175754556 conn(0x557ee7672dc0 msgr2=0x557ee7671fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 shutdown_connections 2024-03-18T15:40:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 1 -- 172.21.15.96:0/2175754556 wait complete. 2024-03-18T15:40:45.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.849+0000 7f8f1679c740 20 client.15678 trim_cache size 0 max 16384 2024-03-18T15:40:45.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:45 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4257473599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 Processor -- start 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 -- start start 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7689150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7671750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a6c80 con 0x557ee7678240 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a87c0 con 0x557ee7677850 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f1679c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a7350 con 0x557ee767d3b0 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f11157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7689150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f11958640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f11958640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51506/0 (socket says 172.21.15.96:51506) 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f11958640 1 -- 172.21.15.96:0/1557391558 learned_addr learned my addr 172.21.15.96:0/1557391558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.875+0000 7f8f12159640 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7671750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 -- 172.21.15.96:0/1557391558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 msgr2=0x557ee7671750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7671750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 -- 172.21.15.96:0/1557391558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 msgr2=0x557ee7689150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7689150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 -- 172.21.15.96:0/1557391558 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ee7680dc0 con 0x557ee7677850 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8f11958640 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009900 tx=0x7f8f080057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc247297a93901e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8ef77fe640 1 -- 172.21.15.96:0/1557391558 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08069070 con 0x557ee7677850 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8ef77fe640 1 -- 172.21.15.96:0/1557391558 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0805f070 con 0x557ee7677850 2024-03-18T15:40:45.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.876+0000 7f8ef77fe640 1 -- 172.21.15.96:0/1557391558 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0804c030 con 0x557ee7677850 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 -- 172.21.15.96:0/1557391558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 msgr2=0x557ee7681fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009900 tx=0x7f8f080057a0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 -- 172.21.15.96:0/1557391558 shutdown_connections 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7671750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7689150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 --2- 172.21.15.96:0/1557391558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7681fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 -- 172.21.15.96:0/1557391558 >> 172.21.15.96:0/1557391558 conn(0x557ee7672dc0 msgr2=0x557ee76818d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 -- 172.21.15.96:0/1557391558 shutdown_connections 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.877+0000 7f8f1679c740 1 -- 172.21.15.96:0/1557391558 wait complete. 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.878+0000 7f8f1679c740 1 -- start start 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.878+0000 7f8f1679c740 1 Processor -- start 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.878+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7686550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7686a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7687a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7686a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a80f0 con 0x557ee7678240 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a7a20 con 0x557ee7677850 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0c1a4c60 con 0x557ee767d3b0 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7686a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54250/0 (socket says 172.21.15.96:54250) 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11157640 1 -- 172.21.15.96:0/356411161 learned_addr learned my addr 172.21.15.96:0/356411161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f1679c740 10 client.0 _ll_register_callbacks cb 0x7f8efc01d520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:45.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7686550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f12159640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7687a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 -- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 msgr2=0x557ee7687a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7687a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 -- 172.21.15.96:0/356411161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 msgr2=0x557ee7686a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7686a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 -- 172.21.15.96:0/356411161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ee73ba7d0 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8f11958640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7686550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802c7b0 tx=0x7f8f0802c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae8aba3f719045 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8ef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.879+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08069070 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8f08048640 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0805f070 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f08075050 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 10 client.25475 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557ee7689150 con 0x557ee7677850 2024-03-18T15:40:45.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0805f070 con 0x557ee7677850 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8f08051500 con 0x557ee7677850 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 10 client.25475 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 10 client.25475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 20 client.25475 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee76f3aa0 0x557ee76f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 4 client.25475 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ee75e3d20 con 0x557ee76f3aa0 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.880+0000 7f8f11157640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee76f3aa0 0x557ee76f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8ef57fa640 10 client.25475 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f11157640 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee76f3aa0 0x557ee76f5e90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f203bb8dde2463d server_cookie=c01ef93a62800e52 in_seq=0 out_seq=0 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8ef57fa640 1 -- 172.21.15.96:0/356411161 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ee75e3d20 con 0x557ee76f3aa0 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8ef57fa640 10 client.25475 handle_command_reply: tid=0 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 client.25475 shutdown 2024-03-18T15:40:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 msgr2=0x557ee7686550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7686550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802c7b0 tx=0x7f8f0802c880 comp rx=0 tx=0).stop 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 shutdown_connections 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557ee767d3b0 0x557ee7687a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557ee76f3aa0 0x557ee76f5e90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ee7678240 0x557ee7686a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8ef57fa640 0 client.25475 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 --2- 172.21.15.96:0/356411161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ee7677850 0x557ee7686550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 >> 172.21.15.96:0/356411161 conn(0x557ee7672dc0 msgr2=0x557ee76768b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 shutdown_connections 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.881+0000 7f8f1679c740 1 -- 172.21.15.96:0/356411161 wait complete. 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f1679c740 20 client.25475 trim_cache size 0 max 16384 2024-03-18T15:40:45.887 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:40:45.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 msgr2=0x7f8ef80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 0x7f8ef80a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8f08000c00 tx=0x7f8f08005fd0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 msgr2=0x7f8f0c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:45.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00b2f0 tx=0x7f8efc00b7b0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 shutdown_connections 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8ef80a2890 0x7f8ef80a4d50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f0c1033c0 0x7f8f0c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f0c102a00 0x7f8f0c1980a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 --2- 172.21.15.96:0/2882921298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c101810 0x7f8f0c197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.882+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 >> 172.21.15.96:0/2882921298 conn(0x7f8f0c0fcd80 msgr2=0x7f8f0c105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.883+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 shutdown_connections 2024-03-18T15:40:45.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:45.883+0000 7f8f13bbc640 1 -- 172.21.15.96:0/2882921298 wait complete. 2024-03-18T15:40:45.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:45 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4257473599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:40:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:46 smithi137 ceph-mon[35868]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:46.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:40:46.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:40:46.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:40:46.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:46 smithi193 ceph-mon[36702]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:46.931 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:46 smithi096 ceph-mon[31930]: pgmap v372: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:47.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 -- 172.21.15.96:0/3516063087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 msgr2=0x7f5d88107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3516063087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d88107040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009860 tx=0x7f5d7802bb70 comp rx=0 tx=0).stop 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 -- 172.21.15.96:0/3516063087 shutdown_connections 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3516063087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88107580 0x7f5d88109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3516063087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d88107040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3516063087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88102320 0x7f5d88104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 -- 172.21.15.96:0/3516063087 >> 172.21.15.96:0/3516063087 conn(0x7f5d880fbc70 msgr2=0x7f5d880fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 -- 172.21.15.96:0/3516063087 shutdown_connections 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.063+0000 7f5d8f852640 1 -- 172.21.15.96:0/3516063087 wait complete. 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 Processor -- start 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 -- start start 2024-03-18T15:40:47.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 0x7f5d88197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d881980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 0x7f5d8819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810d440 con 0x7f5d88107580 2024-03-18T15:40:47.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810d2c0 con 0x7f5d88102320 2024-03-18T15:40:47.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8f852640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8810d5c0 con 0x7f5d88104c50 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8ddef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 0x7f5d8819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 0x7f5d88197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8cded640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d881980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8ddef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 0x7f5d8819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54280/0 (socket says 172.21.15.96:54280) 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.064+0000 7f5d8ddef640 1 -- 172.21.15.96:0/3577397605 learned_addr learned my addr 172.21.15.96:0/3577397605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 -- 172.21.15.96:0/3577397605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 msgr2=0x7f5d88197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 0x7f5d88197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 -- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 msgr2=0x7f5d8819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 0x7f5d8819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 -- 172.21.15.96:0/3577397605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8819f790 con 0x7f5d88104c50 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 0x7f5d88197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d881980b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d7802c0b0 tx=0x7f5d78032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d7e7fc640 1 -- 172.21.15.96:0/3577397605 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78041020 con 0x7f5d88104c50 2024-03-18T15:40:47.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d7e7fc640 1 -- 172.21.15.96:0/3577397605 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7802c9f0 con 0x7f5d88104c50 2024-03-18T15:40:47.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8819fa20 con 0x7f5d88104c50 2024-03-18T15:40:47.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d7e7fc640 1 -- 172.21.15.96:0/3577397605 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78039040 con 0x7f5d88104c50 2024-03-18T15:40:47.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.065+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8819ff60 con 0x7f5d88104c50 2024-03-18T15:40:47.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.068+0000 7f5d7e7fc640 1 -- 172.21.15.96:0/3577397605 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f5d78044020 con 0x7f5d88104c50 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.068+0000 7f5d9243f740 1 Processor -- start 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 -- start start 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 0x561842c4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c52980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561842baf0d0 con 0x561842c4b380 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561842b57c70 con 0x561842c4a990 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d9243f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561842bab170 con 0x561842c504f0 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d7e7fc640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 0x7f5d740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51556/0 (socket says 172.21.15.96:51556) 2024-03-18T15:40:47.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 -- 172.21.15.96:0/3863440955 learned_addr learned my addr 172.21.15.96:0/3863440955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 0x7f5d740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 0x561842c4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 -- 172.21.15.96:0/3863440955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 msgr2=0x561842c52980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c52980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 -- 172.21.15.96:0/3863440955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 msgr2=0x561842c4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d7e7fc640 1 -- 172.21.15.96:0/3577397605 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d780f02f0 con 0x7f5d88104c50 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 0x561842c4fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8ddef640 1 -- 172.21.15.96:0/3863440955 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56184298b7e0 con 0x561842c4a990 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.069+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 0x561842c4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.070+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d80013f50 tx=0x7f5d80011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c8529ae630fbf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.070+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 0x7f5d740a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d80011d70 tx=0x7f5d80008000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.070+0000 7f5d737fe640 1 -- 172.21.15.96:0/3863440955 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80024070 con 0x561842c4a990 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.070+0000 7f5d737fe640 1 -- 172.21.15.96:0/3863440955 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8000edf0 con 0x561842c4a990 2024-03-18T15:40:47.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.070+0000 7f5d737fe640 1 -- 172.21.15.96:0/3863440955 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80020a50 con 0x561842c4a990 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 -- 172.21.15.96:0/3863440955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 msgr2=0x561842c4adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5d80013f50 tx=0x7f5d80011870 comp rx=0 tx=0).stop 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 -- 172.21.15.96:0/3863440955 shutdown_connections 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c52980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4b380 0x561842c4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3863440955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4a990 0x561842c4adb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 -- 172.21.15.96:0/3863440955 >> 172.21.15.96:0/3863440955 conn(0x561842c12c50 msgr2=0x561842c46300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 -- 172.21.15.96:0/3863440955 shutdown_connections 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.071+0000 7f5d9243f740 1 -- 172.21.15.96:0/3863440955 wait complete. 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 -- start start 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 Processor -- start 2024-03-18T15:40:47.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561842c53440 con 0x561842c4a990 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561842b6b0d0 con 0x561842c4b380 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561842c535c0 con 0x561842c504f0 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.072+0000 7f5d9243f740 10 client.0 _ll_register_callbacks cb 0x561842b34760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54294/0 (socket says 172.21.15.96:54294) 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8d5ee640 1 -- 172.21.15.96:0/2829193544 learned_addr learned my addr 172.21.15.96:0/2829193544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 -- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 msgr2=0x561842c5c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 -- 172.21.15.96:0/2829193544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 msgr2=0x561842c54fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 -- 172.21.15.96:0/2829193544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618429ebf70 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d8cded640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c554e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d780edd80 tx=0x7f5d780bc810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88ff048c1943ebb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780cf610 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8000edf0 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d780baf10 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d780bd310 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.073+0000 7f5d9243f740 10 client.25487 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561842883170 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d8000edf0 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d7803c030 con 0x561842c4b380 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 10 client.25487 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 20 client.25487 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842c608e0 0x561842c62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 4 client.25487 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.074+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561842bbc610 con 0x561842c608e0 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.075+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842c608e0 0x561842c62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.075+0000 7f5d70ff9640 10 client.25487 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.075+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842c608e0 0x561842c62cd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec60fa76f4371000 server_cookie=6eb850e852b2a71 in_seq=0 out_seq=0 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d70ff9640 1 -- 172.21.15.96:0/2829193544 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561842bbc610 con 0x561842c608e0 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d70ff9640 10 client.25487 handle_command_reply: tid=0 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 client.25487 shutdown 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 msgr2=0x561842c554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c554e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d780edd80 tx=0x7f5d780bc810 comp rx=0 tx=0).stop 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 shutdown_connections 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c554e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842c608e0 0x561842c62cd0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2829193544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c54fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d70ff9640 0 client.25487 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 >> 172.21.15.96:0/2829193544 conn(0x561842c12c50 msgr2=0x561842c454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 shutdown_connections 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.076+0000 7f5d9243f740 1 -- 172.21.15.96:0/2829193544 wait complete. 2024-03-18T15:40:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.077+0000 7f5d9243f740 20 client.25487 trim_cache size 0 max 16384 2024-03-18T15:40:47.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 Processor -- start 2024-03-18T15:40:47.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 -- start start 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 0x561842c49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881aae70 con 0x561842c4a990 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881ac9b0 con 0x561842c504f0 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d9243f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881ab540 con 0x561842c4b380 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8cded640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c49540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54306/0 (socket says 172.21.15.96:54306) 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8d5ee640 1 -- 172.21.15.96:0/2741354713 learned_addr learned my addr 172.21.15.96:0/2741354713 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 0x561842c49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8cded640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39984/0 (socket says 172.21.15.96:39984) 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 -- 172.21.15.96:0/2741354713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 msgr2=0x561842c5c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 -- 172.21.15.96:0/2741354713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 msgr2=0x561842c49540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c49540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8ddef640 1 -- 172.21.15.96:0/2741354713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561842c4a5d0 con 0x561842c504f0 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.111+0000 7f5d8cded640 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.112+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 0x561842c49a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009270 tx=0x7f5d80009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f04b99802572177 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.112+0000 7f5d72ffd640 1 -- 172.21.15.96:0/2741354713 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80024050 con 0x561842c504f0 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.112+0000 7f5d72ffd640 1 -- 172.21.15.96:0/2741354713 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d80013a60 con 0x561842c504f0 2024-03-18T15:40:47.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.112+0000 7f5d72ffd640 1 -- 172.21.15.96:0/2741354713 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003c60 con 0x561842c504f0 2024-03-18T15:40:47.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- 172.21.15.96:0/2741354713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 msgr2=0x561842c49a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 0x561842c49a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009270 tx=0x7f5d80009f40 comp rx=0 tx=0).stop 2024-03-18T15:40:47.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- 172.21.15.96:0/2741354713 shutdown_connections 2024-03-18T15:40:47.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c504f0 0x561842c49a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c4b380 0x561842c5c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 --2- 172.21.15.96:0/2741354713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c49540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- 172.21.15.96:0/2741354713 >> 172.21.15.96:0/2741354713 conn(0x561842c12c50 msgr2=0x561842c4c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- 172.21.15.96:0/2741354713 shutdown_connections 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- 172.21.15.96:0/2741354713 wait complete. 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 -- start start 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.113+0000 7f5d9243f740 1 Processor -- start 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881ac2e0 con 0x561842c4a990 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881abc10 con 0x561842c4b380 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d881a8e50 con 0x561842c504f0 2024-03-18T15:40:47.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d9243f740 10 client.0 _ll_register_callbacks cb 0x7f5d780bcab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8d5ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51582/0 (socket says 172.21.15.96:51582) 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 -- 172.21.15.96:0/3984826210 learned_addr learned my addr 172.21.15.96:0/3984826210 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8ddef640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 -- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 msgr2=0x561842c5a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 -- 172.21.15.96:0/3984826210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 msgr2=0x561842c5bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c5bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.114+0000 7f5d8cded640 1 -- 172.21.15.96:0/3984826210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561842c5af40 con 0x561842c4b380 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d8cded640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d780b9070 tx=0x7f5d780c52b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf05d10531e2cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780bcc10 con 0x561842c4b380 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d780cf3a0 con 0x561842c4b380 2024-03-18T15:40:47.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d80013a60 con 0x561842c4b380 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d780c0d60 con 0x561842c4b380 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 10 client.25493 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561842cc77d0 con 0x561842c4b380 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d80013a60 con 0x561842c4b380 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d7803c040 con 0x561842c4b380 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 10 client.25493 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 20 client.25493 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842cc7d50 0x561842cca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 4 client.25493 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.115+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561842c54d90 con 0x561842cc7d50 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.116+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842cc7d50 0x561842cca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.116+0000 7f5d70ff9640 10 client.25493 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.116+0000 7f5d8d5ee640 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842cc7d50 0x561842cca170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642880442d598ea3 server_cookie=1dfd6f91db5a569e in_seq=0 out_seq=0 2024-03-18T15:40:47.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d70ff9640 1 -- 172.21.15.96:0/3984826210 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561842c54d90 con 0x561842cc7d50 2024-03-18T15:40:47.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d70ff9640 10 client.25493 handle_command_reply: tid=0 2024-03-18T15:40:47.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 client.25493 shutdown 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 msgr2=0x561842c5c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d780b9070 tx=0x7f5d780c52b0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 shutdown_connections 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561842c504f0 0x561842c5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561842c4b380 0x561842c5c4a0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5d780b9070 tx=0x7f5d780c52b0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561842cc7d50 0x561842cca170 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d70ff9640 0 client.25493 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 --2- 172.21.15.96:0/3984826210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561842c4a990 0x561842c5bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.117+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 >> 172.21.15.96:0/3984826210 conn(0x561842c12c50 msgr2=0x561842c49540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.118+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 shutdown_connections 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.118+0000 7f5d9243f740 1 -- 172.21.15.96:0/3984826210 wait complete. 2024-03-18T15:40:47.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.118+0000 7f5d9243f740 20 client.25493 trim_cache size 0 max 16384 2024-03-18T15:40:47.123 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:40:47.123 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:40:47.123 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "0b1824ef-59ad-48b0-9fb4-18be224ae967", 2024-03-18T15:40:47.123 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:40:47.123 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 msgr2=0x7f5d740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 0x7f5d740a4dd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d80011d70 tx=0x7f5d80008000 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 msgr2=0x7f5d881980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d881980b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d7802c0b0 tx=0x7f5d78032430 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 shutdown_connections 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d740a2910 0x7f5d740a4dd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d88107580 0x7f5d8819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d88104c50 0x7f5d881980b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 --2- 172.21.15.96:0/3577397605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d88102320 0x7f5d88197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 >> 172.21.15.96:0/3577397605 conn(0x7f5d880fbc70 msgr2=0x7f5d88102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 shutdown_connections 2024-03-18T15:40:47.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.119+0000 7f5d8f852640 1 -- 172.21.15.96:0/3577397605 wait complete. 2024-03-18T15:40:47.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:40:47.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:40:47.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 -- 172.21.15.96:0/308526955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc102290 msgr2=0x7f90bc108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 --2- 172.21.15.96:0/308526955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc102290 0x7f90bc108b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009ca0 tx=0x7f90b402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 -- 172.21.15.96:0/308526955 shutdown_connections 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 --2- 172.21.15.96:0/308526955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc102290 0x7f90bc108b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 --2- 172.21.15.96:0/308526955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 --2- 172.21.15.96:0/308526955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc1006e0 0x7f90bc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 -- 172.21.15.96:0/308526955 >> 172.21.15.96:0/308526955 conn(0x7f90bc0fbc70 msgr2=0x7f90bc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 -- 172.21.15.96:0/308526955 shutdown_connections 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.275+0000 7f90c33eb640 1 -- 172.21.15.96:0/308526955 wait complete. 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 Processor -- start 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 -- start start 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 0x7f90bc069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc102290 0x7f90bc19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10c340 con 0x7f90bc102290 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10c1c0 con 0x7f90bc1006e0 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c33eb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10c4c0 con 0x7f90bc1018d0 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 0x7f90bc069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c1988640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc102290 0x7f90bc19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 0x7f90bc069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51598/0 (socket says 172.21.15.96:51598) 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c1187640 1 -- 172.21.15.96:0/3088467223 learned_addr learned my addr 172.21.15.96:0/3088467223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 -- 172.21.15.96:0/3088467223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 msgr2=0x7f90bc069510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 0x7f90bc069510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 -- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc102290 msgr2=0x7f90bc19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc102290 0x7f90bc19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.276+0000 7f90c0986640 1 -- 172.21.15.96:0/3088467223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc19f6d0 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90c0986640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc19c7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00e9f0 tx=0x7f90ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90b27fc640 1 -- 172.21.15.96:0/3088467223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00ccc0 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc19f9c0 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc19ff60 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90b27fc640 1 -- 172.21.15.96:0/3088467223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90ac010910 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.277+0000 7f90b27fc640 1 -- 172.21.15.96:0/3088467223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac018960 con 0x7f90bc1018d0 2024-03-18T15:40:47.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 Processor -- start 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 -- start start 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1809510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a1819ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6a176e500 con 0x55f6a18090f0 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f6a1716be0 con 0x55f6a180cc40 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f6a176a5a0 con 0x55f6a1809ae0 2024-03-18T15:40:47.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1988640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1809510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:40028/0 (socket says 172.21.15.96:40028) 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 -- 172.21.15.96:0/2594614555 learned_addr learned my addr 172.21.15.96:0/2594614555 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90b27fc640 1 -- 172.21.15.96:0/3088467223 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f90ac010ac0 con 0x7f90bc1018d0 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 -- 172.21.15.96:0/2594614555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 msgr2=0x55f6a1819ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a1819ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 -- 172.21.15.96:0/2594614555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 msgr2=0x55f6a1809510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1809510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1187640 1 -- 172.21.15.96:0/2594614555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a172a4b0 con 0x55f6a1809ae0 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.280+0000 7f90c1988640 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1809510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:47.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90c1187640 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d60 tx=0x7f90b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc4fa3206f89ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90b27fc640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 0x7f90a80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90a77fe640 1 -- 172.21.15.96:0/2594614555 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004230 con 0x55f6a1809ae0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90a77fe640 1 -- 172.21.15.96:0/2594614555 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b80043d0 con 0x55f6a1809ae0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90b27fc640 1 -- 172.21.15.96:0/3088467223 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f90ac0cef30 con 0x7f90bc1018d0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90c1988640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 0x7f90a80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.281+0000 7f90a77fe640 1 -- 172.21.15.96:0/2594614555 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8004230 con 0x55f6a1809ae0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- 172.21.15.96:0/2594614555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 msgr2=0x55f6a180c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009d60 tx=0x7f90b8011810 comp rx=0 tx=0).stop 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c1988640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 0x7f90a80a4dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005e30 tx=0x7f90b4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- 172.21.15.96:0/2594614555 shutdown_connections 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a1819ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a180c5d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/2594614555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1809510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- 172.21.15.96:0/2594614555 >> 172.21.15.96:0/2594614555 conn(0x55f6a18046e0 msgr2=0x55f6a1804ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- 172.21.15.96:0/2594614555 shutdown_connections 2024-03-18T15:40:47.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- 172.21.15.96:0/2594614555 wait complete. 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 -- start start 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.282+0000 7f90c5fcd740 1 Processor -- start 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a181c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a1823450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6a181a500 con 0x55f6a1809ae0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f6a172a160 con 0x55f6a18090f0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f6a181a680 con 0x55f6a180cc40 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c5fcd740 10 client.0 _ll_register_callbacks cb 0x55f6a170e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51622/0 (socket says 172.21.15.96:51622) 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 -- 172.21.15.96:0/339331356 learned_addr learned my addr 172.21.15.96:0/339331356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1988640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a1823450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c0986640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a181c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 -- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 msgr2=0x55f6a1823450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a1823450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 -- 172.21.15.96:0/339331356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 msgr2=0x55f6a181c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a181c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 -- 172.21.15.96:0/339331356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a154c7d0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c0986640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a181c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90c1187640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90b801b040 tx=0x7f90b80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2a700687997aef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.283+0000 7f90a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8028020 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b80043d0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f90b80039f0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90b8033050 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 10 client.25502 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6a14823f0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b80043d0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f90b8004bd0 con 0x55f6a18090f0 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 10 client.25502 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 10 client.25502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:47.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 20 client.25502 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a1829c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 4 client.25502 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a177ba40 con 0x55f6a18277d0 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.284+0000 7f90c0986640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a1829c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.285+0000 7f90a4ff9640 10 client.25502 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.285+0000 7f90c0986640 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a1829c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767bb2930c249945 server_cookie=7b2c9dadef9764e3 in_seq=0 out_seq=0 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90a4ff9640 1 -- 172.21.15.96:0/339331356 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55f6a177ba40 con 0x55f6a18277d0 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90a4ff9640 10 client.25502 handle_command_reply: tid=0 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 client.25502 shutdown 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 msgr2=0x55f6a181bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f90b801b040 tx=0x7f90b80035f0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 shutdown_connections 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a1823450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a1829c20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90a4ff9640 0 client.25502 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a181c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/339331356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a181bed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 >> 172.21.15.96:0/339331356 conn(0x55f6a18046e0 msgr2=0x55f6a1803900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 shutdown_connections 2024-03-18T15:40:47.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 1 -- 172.21.15.96:0/339331356 wait complete. 2024-03-18T15:40:47.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.286+0000 7f90c5fcd740 20 client.25502 trim_cache size 0 max 16384 2024-03-18T15:40:47.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.303+0000 7f90c5fcd740 1 Processor -- start 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 -- start start 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a1823410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a18035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a6bd0 con 0x55f6a1809ae0 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a8710 con 0x55f6a18090f0 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c5fcd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a72a0 con 0x55f6a180cc40 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c0986640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a1823410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51634/0 (socket says 172.21.15.96:51634) 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c0986640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a1823410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54350/0 (socket says 172.21.15.96:54350) 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 -- 172.21.15.96:0/1628626553 learned_addr learned my addr 172.21.15.96:0/1628626553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1988640 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a18035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 -- 172.21.15.96:0/1628626553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 msgr2=0x55f6a18035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a18035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 -- 172.21.15.96:0/1628626553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 msgr2=0x55f6a1823410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a1823410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 -- 172.21.15.96:0/1628626553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a14420f0 con 0x55f6a18090f0 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.304+0000 7f90c1187640 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f90b80058c0 tx=0x7f90b8009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50796aa564ad1f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.305+0000 7f90a6ffd640 1 -- 172.21.15.96:0/1628626553 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8028040 con 0x55f6a18090f0 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.305+0000 7f90a6ffd640 1 -- 172.21.15.96:0/1628626553 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b8005510 con 0x55f6a18090f0 2024-03-18T15:40:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.305+0000 7f90a6ffd640 1 -- 172.21.15.96:0/1628626553 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8003c50 con 0x55f6a18090f0 2024-03-18T15:40:47.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- 172.21.15.96:0/1628626553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 msgr2=0x55f6a1803070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f90b80058c0 tx=0x7f90b8009be0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- 172.21.15.96:0/1628626553 shutdown_connections 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a180cc40 0x55f6a18035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a1809ae0 0x55f6a1823410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/1628626553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a18090f0 0x55f6a1803070 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- 172.21.15.96:0/1628626553 >> 172.21.15.96:0/1628626553 conn(0x55f6a18046e0 msgr2=0x55f6a1804ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- 172.21.15.96:0/1628626553 shutdown_connections 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- 172.21.15.96:0/1628626553 wait complete. 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 -- start start 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.306+0000 7f90c5fcd740 1 Processor -- start 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a18208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54352/0 (socket says 172.21.15.96:54352) 2024-03-18T15:40:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 -- 172.21.15.96:0/3777204993 learned_addr learned my addr 172.21.15.96:0/3777204993 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a188bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a8040 con 0x55f6a18090f0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a7970 con 0x55f6a180cc40 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1a4bb0 con 0x55f6a1809ae0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c0986640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a18208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c5fcd740 10 client.0 _ll_register_callbacks cb 0x7f90b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1988640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a188bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 -- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 msgr2=0x55f6a18208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a18208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 -- 172.21.15.96:0/3777204993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 msgr2=0x55f6a188bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a188bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c1187640 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a17359d0 con 0x55f6a18090f0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.307+0000 7f90c0986640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a18208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90c1187640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90b8025730 tx=0x7f90b8025760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9353d4187bc63b3f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8044060 con 0x55f6a18090f0 2024-03-18T15:40:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f90b8025d90 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b8005510 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.308+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90b8028040 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f6a188d1c0 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b8005510 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f90b803f920 con 0x55f6a18090f0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 20 client.15717 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a188fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 4 client.15717 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6a1775d90 con 0x55f6a18277d0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.309+0000 7f90c0986640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a188fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90a4ff9640 10 client.15717 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c0986640 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a188fa00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef44119c7eb3f44d server_cookie=6a5ae480c13583fb in_seq=0 out_seq=0 2024-03-18T15:40:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90a4ff9640 1 -- 172.21.15.96:0/3777204993 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f6a1775d90 con 0x55f6a18277d0 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90a4ff9640 10 client.15717 handle_command_reply: tid=0 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 client.15717 shutdown 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 msgr2=0x55f6a1820390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90b8025730 tx=0x7f90b8025760 comp rx=0 tx=0).stop 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 shutdown_connections 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6a180cc40 0x55f6a188bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f6a1809ae0 0x55f6a18208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55f6a18277d0 0x55f6a188fa00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 --2- 172.21.15.96:0/3777204993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6a18090f0 0x55f6a1820390 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 >> 172.21.15.96:0/3777204993 conn(0x55f6a18046e0 msgr2=0x55f6a1807940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90a4ff9640 0 client.15717 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 shutdown_connections 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.310+0000 7f90c5fcd740 1 -- 172.21.15.96:0/3777204993 wait complete. 2024-03-18T15:40:47.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c5fcd740 20 client.15717 trim_cache size 0 max 16384 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "0b1824ef-59ad-48b0-9fb4-18be224ae967": { 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "0b1824ef-59ad-48b0-9fb4-18be224ae967", 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:40:47.316 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 msgr2=0x7f90a80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 0x7f90a80a4dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f90b4005e30 tx=0x7f90b4032590 comp rx=0 tx=0).stop 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 msgr2=0x7f90bc19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc19c7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00e9f0 tx=0x7f90ac00eeb0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 shutdown_connections 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f90a80a2910 0x7f90a80a4dd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90bc102290 0x7f90bc19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90bc1018d0 0x7f90bc19c7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 --2- 172.21.15.96:0/3088467223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90bc1006e0 0x7f90bc069510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:47.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 >> 172.21.15.96:0/3088467223 conn(0x7f90bc0fbc70 msgr2=0x7f90bc104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:47.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 shutdown_connections 2024-03-18T15:40:47.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:40:47.311+0000 7f90c33eb640 1 -- 172.21.15.96:0/3088467223 wait complete. 2024-03-18T15:40:47.326 INFO:tasks.cephfs.filesystem:scrub status for tag:0b1824ef-59ad-48b0-9fb4-18be224ae967 - {'path': '/', 'tag': '0b1824ef-59ad-48b0-9fb4-18be224ae967', 'options': 'recursive,force'} 2024-03-18T15:40:47.560 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:47 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:40:47.560 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:47 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:47.560 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:47 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:40:47.561 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:47 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:47 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:40:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:47 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:47.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:47 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:40:47.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:47 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:47 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:40:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:47 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:47 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:40:47.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:47 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:40:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:48 smithi137 ceph-mon[35868]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:40:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:48 smithi193 ceph-mon[36702]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:40:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:48 smithi096 ceph-mon[31930]: pgmap v373: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-18T15:40:49.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:49 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:40:49.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:49 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:40:49.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:49 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:40:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:50 smithi137 ceph-mon[35868]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:50 smithi193 ceph-mon[36702]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:50 smithi096 ceph-mon[31930]: pgmap v374: 97 pgs: 97 active+clean; 660 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-18T15:40:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:52 smithi137 ceph-mon[35868]: pgmap v375: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T15:40:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:52 smithi193 ceph-mon[36702]: pgmap v375: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T15:40:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:52 smithi096 ceph-mon[31930]: pgmap v375: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.548+0000 7f637b5b9640 1 -- 172.21.15.96:0/3709295673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 msgr2=0x7f63741046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.548+0000 7f637b5b9640 1 --2- 172.21.15.96:0/3709295673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f63741046b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6370009900 tx=0x7f637002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 -- 172.21.15.96:0/3709295673 shutdown_connections 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 --2- 172.21.15.96:0/3709295673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374107520 0x7f6374109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 --2- 172.21.15.96:0/3709295673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374104bf0 0x7f6374106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 --2- 172.21.15.96:0/3709295673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f63741046b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 -- 172.21.15.96:0/3709295673 >> 172.21.15.96:0/3709295673 conn(0x7f63740fbc70 msgr2=0x7f63740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 -- 172.21.15.96:0/3709295673 shutdown_connections 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 -- 172.21.15.96:0/3709295673 wait complete. 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 Processor -- start 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.549+0000 7f637b5b9640 1 -- start start 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374107520 0x7f637419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d4a0 con 0x7f63741022c0 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d320 con 0x7f6374104bf0 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f637b5b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f637410d620 con 0x7f6374107520 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6378b54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34472/0 (socket says 172.21.15.96:34472) 2024-03-18T15:40:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6378b54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38084/0 (socket says 172.21.15.96:38084) 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 -- 172.21.15.96:0/1722602006 learned_addr learned my addr 172.21.15.96:0/1722602006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379b56640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374107520 0x7f637419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 -- 172.21.15.96:0/1722602006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374107520 msgr2=0x7f637419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374107520 0x7f637419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 -- 172.21.15.96:0/1722602006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 msgr2=0x7f6374198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 -- 172.21.15.96:0/1722602006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637419f6e0 con 0x7f63741022c0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6378b54640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.550+0000 7f6379355640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6370009230 tx=0x7f637002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.551+0000 7f636a7fc640 1 -- 172.21.15.96:0/1722602006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370041070 con 0x7f63741022c0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.551+0000 7f636a7fc640 1 -- 172.21.15.96:0/1722602006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6370004450 con 0x7f63741022c0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.551+0000 7f636a7fc640 1 -- 172.21.15.96:0/1722602006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370003e20 con 0x7f63741022c0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.551+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637419f970 con 0x7f63741022c0 2024-03-18T15:40:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.551+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637419fe10 con 0x7f63741022c0 2024-03-18T15:40:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.554+0000 7f636a7fc640 1 -- 172.21.15.96:0/1722602006 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f6370044070 con 0x7f63741022c0 2024-03-18T15:40:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.554+0000 7f637e1a7740 1 Processor -- start 2024-03-18T15:40:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.554+0000 7f636a7fc640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 0x7f63600a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.554+0000 7f637e1a7740 1 -- start start 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379b56640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 0x7f63600a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f636a7fc640 1 -- 172.21.15.96:0/1722602006 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f63700f0620 con 0x7f63741022c0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5b0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5614ec510a40 con 0x5614ec5ab340 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614ec4b9920 con 0x5614ec5b52b0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f637e1a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614ec50cae0 con 0x5614ec5abd30 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34478/0 (socket says 172.21.15.96:34478) 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 -- 172.21.15.96:0/4154912861 learned_addr learned my addr 172.21.15.96:0/4154912861 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6378b54640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379b56640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 -- 172.21.15.96:0/4154912861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 msgr2=0x5614ec5b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 -- 172.21.15.96:0/4154912861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 msgr2=0x5614ec5b7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5b7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 -- 172.21.15.96:0/4154912861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ec4d1460 con 0x5614ec5ab340 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379b56640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379b56640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 0x7f63600a4e80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f636c005fd0 tx=0x7f636c009f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f6379355640 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f63700b99c0 tx=0x7f637002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d71ebf1aeac1e58 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.555+0000 7f635f7fe640 1 -- 172.21.15.96:0/4154912861 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637004b070 con 0x5614ec5ab340 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f635f7fe640 1 -- 172.21.15.96:0/4154912861 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63700c1090 con 0x5614ec5ab340 2024-03-18T15:40:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f635f7fe640 1 -- 172.21.15.96:0/4154912861 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63700c5250 con 0x5614ec5ab340 2024-03-18T15:40:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 -- 172.21.15.96:0/4154912861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 msgr2=0x5614ec5ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f63700b99c0 tx=0x7f637002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 -- 172.21.15.96:0/4154912861 shutdown_connections 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5b7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4154912861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5ab760 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 -- 172.21.15.96:0/4154912861 >> 172.21.15.96:0/4154912861 conn(0x5614ec5a68b0 msgr2=0x5614ec5a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 -- 172.21.15.96:0/4154912861 shutdown_connections 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.556+0000 7f637e1a7740 1 -- 172.21.15.96:0/4154912861 wait complete. 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 -- start start 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 Processor -- start 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5614ec5b8020 con 0x5614ec5abd30 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5614ec4ccd70 con 0x5614ec5b52b0 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5614ec5b81a0 con 0x5614ec5ab340 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f637e1a7740 10 client.0 _ll_register_callbacks cb 0x5614ec224a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6379355640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6378b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6379355640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:56446/0 (socket says 172.21.15.96:56446) 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6379355640 1 -- 172.21.15.96:0/1333727426 learned_addr learned my addr 172.21.15.96:0/1333727426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6378b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34480/0 (socket says 172.21.15.96:34480) 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.557+0000 7f6379b56640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 -- 172.21.15.96:0/1333727426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 msgr2=0x5614ec5c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 -- 172.21.15.96:0/1333727426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 msgr2=0x5614ec5b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 -- 172.21.15.96:0/1333727426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ec225210 con 0x5614ec5ab340 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6378b54640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f6379355640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63700c51b0 tx=0x7f6370009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab84977a6e01f59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f635cff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637004b030 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f63700bb2f0 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63700c1090 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f63700c56e0 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f637e1a7740 10 client.25570 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614ec2f1620 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.558+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63700c1090 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f63700bc590 con 0x5614ec5ab340 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 10 client.25570 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 10 client.25570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 20 client.25570 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec5c5200 0x5614ec5c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 4 client.25570 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ec51df80 con 0x5614ec5c5200 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f6378b54640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec5c5200 0x5614ec5c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f635cff9640 10 client.25570 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.559+0000 7f6378b54640 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec5c5200 0x5614ec5c75f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b85b1b55ec7e00 server_cookie=bd3e482b92f23007 in_seq=0 out_seq=0 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f635cff9640 1 -- 172.21.15.96:0/1333727426 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5614ec51df80 con 0x5614ec5c5200 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f635cff9640 10 client.25570 handle_command_reply: tid=0 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 client.25570 shutdown 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 msgr2=0x5614ec5b9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f63700c51b0 tx=0x7f6370009260 comp rx=0 tx=0).stop 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 shutdown_connections 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec5c5200 0x5614ec5c75f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5abd30 0x5614ec5b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 --2- 172.21.15.96:0/1333727426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5ab340 0x5614ec5b9790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 >> 172.21.15.96:0/1333727426 conn(0x5614ec5a68b0 msgr2=0x5614ec5a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f635cff9640 0 client.25570 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 shutdown_connections 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.560+0000 7f637e1a7740 1 -- 172.21.15.96:0/1333727426 wait complete. 2024-03-18T15:40:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.561+0000 7f637e1a7740 20 client.25570 trim_cache size 0 max 16384 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.588+0000 7f637e1a7740 1 Processor -- start 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 -- start start 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a6be0 con 0x5614ec5ab340 2024-03-18T15:40:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a8720 con 0x5614ec5b52b0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f637e1a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a72b0 con 0x5614ec5abd30 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34490/0 (socket says 172.21.15.96:34490) 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379355640 1 -- 172.21.15.96:0/4044121840 learned_addr learned my addr 172.21.15.96:0/4044121840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6378b54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 -- 172.21.15.96:0/4044121840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 msgr2=0x5614ec5aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 -- 172.21.15.96:0/4044121840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 msgr2=0x5614ec5b9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379b56640 1 -- 172.21.15.96:0/4044121840 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ec5a7620 con 0x5614ec5b52b0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.589+0000 7f6379355640 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.590+0000 7f6379b56640 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5cbed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f636c00f900 tx=0x7f636c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ed7911e79aa24d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.590+0000 7f635effd640 1 -- 172.21.15.96:0/4044121840 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c017040 con 0x5614ec5b52b0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.590+0000 7f635effd640 1 -- 172.21.15.96:0/4044121840 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c02b070 con 0x5614ec5b52b0 2024-03-18T15:40:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.590+0000 7f635effd640 1 -- 172.21.15.96:0/4044121840 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636c026960 con 0x5614ec5b52b0 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- 172.21.15.96:0/4044121840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 msgr2=0x5614ec5cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5cbed0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f636c00f900 tx=0x7f636c01c000 comp rx=0 tx=0).stop 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- 172.21.15.96:0/4044121840 shutdown_connections 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5b52b0 0x5614ec5cbed0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5abd30 0x5614ec5aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 --2- 172.21.15.96:0/4044121840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5b9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- 172.21.15.96:0/4044121840 >> 172.21.15.96:0/4044121840 conn(0x5614ec5a68b0 msgr2=0x5614ec5a9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- 172.21.15.96:0/4044121840 shutdown_connections 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- 172.21.15.96:0/4044121840 wait complete. 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 -- start start 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.591+0000 7f637e1a7740 1 Processor -- start 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 0x5614ec5bebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5b52b0 0x5614ec5bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a8050 con 0x5614ec5ab340 2024-03-18T15:40:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a7980 con 0x5614ec5abd30 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63741a4bc0 con 0x5614ec5b52b0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f637e1a7740 10 client.0 _ll_register_callbacks cb 0x7f6364009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34506/0 (socket says 172.21.15.96:34506) 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 -- 172.21.15.96:0/3987269223 learned_addr learned my addr 172.21.15.96:0/3987269223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6378b54640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 0x5614ec5bebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379b56640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5b52b0 0x5614ec5bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 -- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5b52b0 msgr2=0x5614ec5bfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5b52b0 0x5614ec5bfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 -- 172.21.15.96:0/3987269223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 msgr2=0x5614ec5bebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 0x5614ec5bebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 -- 172.21.15.96:0/3987269223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ec5c0160 con 0x5614ec5ab340 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6378b54640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 0x5614ec5bebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f6379355640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f637002c7b0 tx=0x7f637002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2a4a455458f31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:40:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.592+0000 7f635cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637004b030 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f63700bb2f0 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636c02b070 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f637009be60 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614ec5c0160 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f636c02b070 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f63700b7b60 con 0x5614ec5ab340 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 20 client.15735 populate_metadata read hostname 'smithi096' 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec62c060 0x5614ec62e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 4 client.15735 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614ec5182d0 con 0x5614ec62c060 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.593+0000 7f6378b54640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec62c060 0x5614ec62e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:40:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.594+0000 7f635cff9640 10 client.15735 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.594+0000 7f6378b54640 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec62c060 0x5614ec62e480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba9be07db4f22b07 server_cookie=dcab172756515ac2 in_seq=0 out_seq=0 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f635cff9640 1 -- 172.21.15.96:0/3987269223 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614ec5182d0 con 0x5614ec62c060 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f635cff9640 10 client.15735 handle_command_reply: tid=0 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 client.15735 shutdown 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 msgr2=0x5614ec5be670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f637002c7b0 tx=0x7f637002bc30 comp rx=0 tx=0).stop 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 shutdown_connections 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5614ec5b52b0 0x5614ec5bfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5614ec5abd30 0x5614ec5bebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5614ec62c060 0x5614ec62e480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 --2- 172.21.15.96:0/3987269223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5614ec5ab340 0x5614ec5be670 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 >> 172.21.15.96:0/3987269223 conn(0x5614ec5a68b0 msgr2=0x5614ec5aa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f635cff9640 0 client.15735 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 shutdown_connections 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 1 -- 172.21.15.96:0/3987269223 wait complete. 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.601+0000 7f637e1a7740 20 client.15735 trim_cache size 0 max 16384 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:40:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:40:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 msgr2=0x7f63600a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 0x7f63600a4e80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f636c005fd0 tx=0x7f636c009f90 comp rx=0 tx=0).stop 2024-03-18T15:40:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 msgr2=0x7f6374197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6370009230 tx=0x7f637002c650 comp rx=0 tx=0).stop 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 shutdown_connections 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f63600a29c0 0x7f63600a4e80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6374107520 0x7f637419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6374104bf0 0x7f6374198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 --2- 172.21.15.96:0/1722602006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63741022c0 0x7f6374197ac0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 >> 172.21.15.96:0/1722602006 conn(0x7f63740fbc70 msgr2=0x7f6374102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 shutdown_connections 2024-03-18T15:40:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:40:53.602+0000 7f637b5b9640 1 -- 172.21.15.96:0/1722602006 wait complete. 2024-03-18T15:40:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:54 smithi137 ceph-mon[35868]: pgmap v376: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:54 smithi193 ceph-mon[36702]: pgmap v376: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:54 smithi096 ceph-mon[31930]: pgmap v376: 97 pgs: 97 active+clean; 669 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-03-18T15:40:56.236 INFO:tasks.workunit.client.0.smithi096.stderr:Note: switching to '4e19f704afd9957ea657ab858f9ecf76401fef6f'. 2024-03-18T15:40:56.236 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.236 INFO:tasks.workunit.client.0.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: git switch -c 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:Or undo this operation with: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: git switch - 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:56.237 INFO:tasks.workunit.client.0.smithi096.stderr:HEAD is now at 4e19f704afd ignorelist mon down 2024-03-18T15:40:56.242 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-18T15:40:56.266 INFO:tasks.workunit.client.0.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-18T15:40:56.268 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T15:40:56.268 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-18T15:40:56.858 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:56 smithi137 ceph-mon[35868]: pgmap v377: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:40:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:56 smithi193 ceph-mon[36702]: pgmap v377: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:40:56.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:56 smithi096 ceph-mon[31930]: pgmap v377: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:40:57.012 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-18T15:40:57.077 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-18T15:40:57.117 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-18T15:40:57.119 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T15:40:57.119 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-18T15:40:57.157 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-18T15:40:57.160 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:40:57.160 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-18T15:40:57.216 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-03-18T15:40:57.217 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-03-18T15:40:57.217 DEBUG:teuthology.orchestra.run.smithi096:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4e19f704afd9957ea657ab858f9ecf76401fef6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-03-18T15:40:57.276 INFO:tasks.workunit.client.0.smithi096.stdout:getting blogbench 2024-03-18T15:40:57.276 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'getting blogbench' 2024-03-18T15:40:57.276 INFO:tasks.workunit.client.0.smithi096.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-03-18T15:40:57.295 INFO:tasks.workunit.client.0.smithi096.stderr:--2024-03-18 15:40:57-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-03-18T15:40:57.295 INFO:tasks.workunit.client.0.smithi096.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-18T15:40:57.331 INFO:tasks.workunit.client.0.smithi096.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-18T15:40:57.370 INFO:tasks.workunit.client.0.smithi096.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-18T15:40:57.371 INFO:tasks.workunit.client.0.smithi096.stderr:Length: 103355 (101K) [application/octet-stream] 2024-03-18T15:40:57.372 INFO:tasks.workunit.client.0.smithi096.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-03-18T15:40:57.374 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:57.450 INFO:tasks.workunit.client.0.smithi096.stderr: 0K .......... .......... .......... .......... .......... 49% 649K 0s 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr: 50K .......... .......... .......... .......... .......... 99% 1.35M 0s 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr: 100K 100% 39.4M=0.1s 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr:2024-03-18 15:40:57 (891 KB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-03-18T15:40:57.487 INFO:tasks.workunit.client.0.smithi096.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-03-18T15:40:57.509 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/ 2024-03-18T15:40:57.510 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/m4/ 2024-03-18T15:40:57.511 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-03-18T15:40:57.513 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/man/ 2024-03-18T15:40:57.514 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/man/blogbench.8 2024-03-18T15:40:57.516 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/man/Makefile.am 2024-03-18T15:40:57.517 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/man/Makefile.in 2024-03-18T15:40:57.519 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/ 2024-03-18T15:40:57.519 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/mysnprintf.c 2024-03-18T15:40:57.521 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/mysnprintf.h 2024-03-18T15:40:57.522 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/blogbench.c 2024-03-18T15:40:57.524 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/blogbench.h 2024-03-18T15:40:57.525 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/fakesnprintf.c 2024-03-18T15:40:57.526 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/fakesnprintf.h 2024-03-18T15:40:57.528 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/blogbench_p.h 2024-03-18T15:40:57.529 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-03-18T15:40:57.531 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-03-18T15:40:57.532 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/Makefile.am 2024-03-18T15:40:57.533 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/Makefile.in 2024-03-18T15:40:57.534 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/process.c 2024-03-18T15:40:57.535 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/globals.h 2024-03-18T15:40:57.536 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/commenter.c 2024-03-18T15:40:57.537 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/rewriter.c 2024-03-18T15:40:57.538 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/reader.c 2024-03-18T15:40:57.539 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/writer.c 2024-03-18T15:40:57.540 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/src/helpers.c 2024-03-18T15:40:57.540 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/NEWS 2024-03-18T15:40:57.541 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/compile 2024-03-18T15:40:57.542 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/depcomp 2024-03-18T15:40:57.543 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/aclocal.m4 2024-03-18T15:40:57.544 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/README 2024-03-18T15:40:57.545 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/configure 2024-03-18T15:40:57.548 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/configure.ac 2024-03-18T15:40:57.549 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/config.guess 2024-03-18T15:40:57.550 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/install-sh 2024-03-18T15:40:57.551 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/config.sub 2024-03-18T15:40:57.552 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/missing 2024-03-18T15:40:57.553 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/mkinstalldirs 2024-03-18T15:40:57.553 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/Makefile.am 2024-03-18T15:40:57.554 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/Makefile.in 2024-03-18T15:40:57.555 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/config.h.in 2024-03-18T15:40:57.556 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/AUTHORS 2024-03-18T15:40:57.557 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/INSTALL 2024-03-18T15:40:57.558 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/ChangeLog 2024-03-18T15:40:57.559 INFO:tasks.workunit.client.0.smithi096.stdout:blogbench-1.0/COPYING 2024-03-18T15:40:57.560 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd blogbench-1.0/ 2024-03-18T15:40:57.560 INFO:tasks.workunit.client.0.smithi096.stdout:making blogbench 2024-03-18T15:40:57.560 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'making blogbench' 2024-03-18T15:40:57.560 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./configure 2024-03-18T15:40:57.672 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-03-18T15:40:58.682 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether build environment is sane... yes 2024-03-18T15:40:58.694 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gawk... gawk 2024-03-18T15:40:58.710 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether make sets $(MAKE)... yes 2024-03-18T15:40:58.730 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc... gcc 2024-03-18T15:40:58.784 INFO:tasks.workunit.client.0.smithi096.stdout:checking for C compiler default output file name... a.out 2024-03-18T15:40:58.786 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether the C compiler works... yes 2024-03-18T15:40:58.788 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are cross compiling... no 2024-03-18T15:40:58.826 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of executables... 2024-03-18T15:40:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:40:58 smithi137 ceph-mon[35868]: pgmap v378: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T15:40:58.865 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of object files... o 2024-03-18T15:40:58.896 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are using the GNU C compiler... yes 2024-03-18T15:40:58.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:40:58 smithi193 ceph-mon[36702]: pgmap v378: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T15:40:58.933 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc accepts -g... yes 2024-03-18T15:40:58.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:40:58 smithi096 ceph-mon[31930]: pgmap v378: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-03-18T15:40:58.989 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to accept ANSI C... none needed 2024-03-18T15:40:59.050 INFO:tasks.workunit.client.0.smithi096.stdout:checking for style of include used by make... GNU 2024-03-18T15:40:59.181 INFO:tasks.workunit.client.0.smithi096.stdout:checking dependency style of gcc... gcc3 2024-03-18T15:40:59.181 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-03-18T15:40:59.250 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to run the C preprocessor... gcc -E 2024-03-18T15:40:59.293 INFO:tasks.workunit.client.0.smithi096.stdout:checking for egrep... grep -E 2024-03-18T15:40:59.302 INFO:tasks.workunit.client.0.smithi096.stdout:checking for AIX... no 2024-03-18T15:40:59.542 INFO:tasks.workunit.client.0.smithi096.stdout:checking for library containing strerror... none required 2024-03-18T15:40:59.549 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ANSI C header files... yes 2024-03-18T15:41:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:00 smithi137 ceph-mon[35868]: pgmap v379: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:41:00.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:00 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:00 smithi193 ceph-mon[36702]: pgmap v379: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:41:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:00 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:00.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:00 smithi096 ceph-mon[31930]: pgmap v379: 97 pgs: 97 active+clean; 670 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-18T15:41:00.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:00 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:02 smithi137 ceph-mon[35868]: pgmap v380: 97 pgs: 97 active+clean; 1.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-03-18T15:41:02.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:02 smithi193 ceph-mon[36702]: pgmap v380: 97 pgs: 97 active+clean; 1.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-03-18T15:41:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:02 smithi096 ceph-mon[31930]: pgmap v380: 97 pgs: 97 active+clean; 1.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 50 KiB/s wr, 14 op/s 2024-03-18T15:41:04.554 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-03-18T15:41:04.624 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/types.h... yes 2024-03-18T15:41:04.680 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/stat.h... yes 2024-03-18T15:41:04.733 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdlib.h... yes 2024-03-18T15:41:04.789 INFO:tasks.workunit.client.0.smithi096.stdout:checking for string.h... yes 2024-03-18T15:41:04.845 INFO:tasks.workunit.client.0.smithi096.stdout:checking for memory.h... yes 2024-03-18T15:41:04.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:04 smithi137 ceph-mon[35868]: pgmap v381: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-03-18T15:41:04.898 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... yes 2024-03-18T15:41:04.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:04 smithi193 ceph-mon[36702]: pgmap v381: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-03-18T15:41:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:04 smithi096 ceph-mon[31930]: pgmap v381: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-03-18T15:41:04.952 INFO:tasks.workunit.client.0.smithi096.stdout:checking for inttypes.h... yes 2024-03-18T15:41:05.007 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdint.h... yes 2024-03-18T15:41:05.066 INFO:tasks.workunit.client.0.smithi096.stdout:checking for unistd.h... yes 2024-03-18T15:41:05.081 INFO:tasks.workunit.client.0.smithi096.stdout:checking for memory.h... (cached) yes 2024-03-18T15:41:05.086 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... (cached) yes 2024-03-18T15:41:05.090 INFO:tasks.workunit.client.0.smithi096.stdout:checking for unistd.h... (cached) yes 2024-03-18T15:41:05.181 INFO:tasks.workunit.client.0.smithi096.stdout:checking locale.h usability... yes 2024-03-18T15:41:05.201 INFO:tasks.workunit.client.0.smithi096.stdout:checking locale.h presence... yes 2024-03-18T15:41:05.201 INFO:tasks.workunit.client.0.smithi096.stdout:checking for locale.h... yes 2024-03-18T15:41:05.220 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdlib.h... (cached) yes 2024-03-18T15:41:05.224 INFO:tasks.workunit.client.0.smithi096.stdout:checking for string.h... (cached) yes 2024-03-18T15:41:05.296 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/socket.h usability... yes 2024-03-18T15:41:05.322 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/socket.h presence... yes 2024-03-18T15:41:05.323 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/socket.h... yes 2024-03-18T15:41:05.381 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h usability... yes 2024-03-18T15:41:05.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h presence... yes 2024-03-18T15:41:05.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/time.h... yes 2024-03-18T15:41:05.463 INFO:tasks.workunit.client.0.smithi096.stdout:checking fcntl.h usability... yes 2024-03-18T15:41:05.483 INFO:tasks.workunit.client.0.smithi096.stdout:checking fcntl.h presence... yes 2024-03-18T15:41:05.483 INFO:tasks.workunit.client.0.smithi096.stdout:checking for fcntl.h... yes 2024-03-18T15:41:05.548 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h usability... yes 2024-03-18T15:41:05.568 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h presence... yes 2024-03-18T15:41:05.569 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fcntl.h... yes 2024-03-18T15:41:05.656 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h usability... yes 2024-03-18T15:41:05.682 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h presence... yes 2024-03-18T15:41:05.682 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/param.h... yes 2024-03-18T15:41:05.732 INFO:tasks.workunit.client.0.smithi096.stdout:checking ioctl.h usability... no 2024-03-18T15:41:05.757 INFO:tasks.workunit.client.0.smithi096.stdout:checking ioctl.h presence... no 2024-03-18T15:41:05.758 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ioctl.h... no 2024-03-18T15:41:05.814 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h usability... yes 2024-03-18T15:41:05.834 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h presence... yes 2024-03-18T15:41:05.835 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/ioctl.h... yes 2024-03-18T15:41:05.895 INFO:tasks.workunit.client.0.smithi096.stdout:checking return type of signal handlers... void 2024-03-18T15:41:05.939 INFO:tasks.workunit.client.0.smithi096.stdout:checking for size_t... yes 2024-03-18T15:41:05.982 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dev_t... yes 2024-03-18T15:41:06.032 INFO:tasks.workunit.client.0.smithi096.stdout:checking for short... yes 2024-03-18T15:41:06.125 INFO:tasks.workunit.client.0.smithi096.stdout:checking size of short... 2 2024-03-18T15:41:06.170 INFO:tasks.workunit.client.0.smithi096.stdout:checking for int... yes 2024-03-18T15:41:06.249 INFO:tasks.workunit.client.0.smithi096.stdout:checking size of int... 4 2024-03-18T15:41:06.293 INFO:tasks.workunit.client.0.smithi096.stdout:checking for long... yes 2024-03-18T15:41:06.372 INFO:tasks.workunit.client.0.smithi096.stdout:checking size of long... 8 2024-03-18T15:41:06.420 INFO:tasks.workunit.client.0.smithi096.stdout:checking for long long... yes 2024-03-18T15:41:06.501 INFO:tasks.workunit.client.0.smithi096.stdout:checking size of long long... 8 2024-03-18T15:41:06.551 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ssize_t... yes 2024-03-18T15:41:06.594 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sig_atomic_t... yes 2024-03-18T15:41:06.652 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc needs -traditional... no 2024-03-18T15:41:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 -- 172.21.15.96:0/628328546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078100730 msgr2=0x7f1078100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 --2- 172.21.15.96:0/628328546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078100730 0x7f1078100b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1068009900 tx=0x7f106802bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 -- 172.21.15.96:0/628328546 shutdown_connections 2024-03-18T15:41:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 --2- 172.21.15.96:0/628328546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f1078108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 --2- 172.21.15.96:0/628328546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078101920 0x7f1078101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 --2- 172.21.15.96:0/628328546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078100730 0x7f1078100b50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 -- 172.21.15.96:0/628328546 >> 172.21.15.96:0/628328546 conn(0x7f10780fbcc0 msgr2=0x7f10780fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 -- 172.21.15.96:0/628328546 shutdown_connections 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.691+0000 7f107e748640 1 -- 172.21.15.96:0/628328546 wait complete. 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 Processor -- start 2024-03-18T15:41:06.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 -- start start 2024-03-18T15:41:06.693 INFO:tasks.workunit.client.0.smithi096.stdout:checking for an ANSI C-conforming const... yes 2024-03-18T15:41:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078100730 0x7f1078197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f107819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f107810c380 con 0x7f1078101920 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f107810c200 con 0x7f10781022e0 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107e748640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f107810c500 con 0x7f1078100730 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55632/0 (socket says 172.21.15.96:55632) 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f1077fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078100730 0x7f1078197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 -- 172.21.15.96:0/4152065392 learned_addr learned my addr 172.21.15.96:0/4152065392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107cce5640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f107819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 -- 172.21.15.96:0/4152065392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078100730 msgr2=0x7f1078197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078100730 0x7f1078197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 -- 172.21.15.96:0/4152065392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 msgr2=0x7f107819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f107819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f10777fe640 1 -- 172.21.15.96:0/4152065392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107819f6b0 con 0x7f1078101920 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.692+0000 7f107cce5640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f107819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f10777fe640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f10757fa640 1 -- 172.21.15.96:0/4152065392 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c017030 con 0x7f1078101920 2024-03-18T15:41:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107819f9a0 con 0x7f1078101920 2024-03-18T15:41:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f10757fa640 1 -- 172.21.15.96:0/4152065392 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c004020 con 0x7f1078101920 2024-03-18T15:41:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f10757fa640 1 -- 172.21.15.96:0/4152065392 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c0047a0 con 0x7f1078101920 2024-03-18T15:41:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.693+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107819fef0 con 0x7f1078101920 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.695+0000 7f10757fa640 1 -- 172.21.15.96:0/4152065392 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f106c0041d0 con 0x7f1078101920 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.695+0000 7f108140c740 1 Processor -- start 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.695+0000 7f108140c740 1 -- start start 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10757fa640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 0x7f10600a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c2c90fa40 con 0x559c2c9af140 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c2c8b8920 con 0x559c2c9b42b0 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f108140c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c2c90bae0 con 0x559c2c9ae750 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f1077fff640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 0x7f10600a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10757fa640 1 -- 172.21.15.96:0/4152065392 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f106c0ce620 con 0x7f1078101920 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55648/0 (socket says 172.21.15.96:55648) 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 -- 172.21.15.96:0/3434823364 learned_addr learned my addr 172.21.15.96:0/3434823364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f107cce5640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f107cce5640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 -- 172.21.15.96:0/3434823364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 msgr2=0x559c2c9aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 -- 172.21.15.96:0/3434823364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 msgr2=0x559c2c9b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 -- 172.21.15.96:0/3434823364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2c8d0460 con 0x559c2c9af140 2024-03-18T15:41:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f107cce5640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f1077fff640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 0x7f10600a4e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1068000c00 tx=0x7f1068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f10777fe640 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f106c004770 tx=0x7f106c016c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd8760db0fd6364 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.696+0000 7f105e7fc640 1 -- 172.21.15.96:0/3434823364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c09a730 con 0x559c2c9af140 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f105e7fc640 1 -- 172.21.15.96:0/3434823364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c09b4d0 con 0x559c2c9af140 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f105e7fc640 1 -- 172.21.15.96:0/3434823364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c098420 con 0x559c2c9af140 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 -- 172.21.15.96:0/3434823364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 msgr2=0x559c2c9b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f106c004770 tx=0x7f106c016c70 comp rx=0 tx=0).stop 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 -- 172.21.15.96:0/3434823364 shutdown_connections 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b3c40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 --2- 172.21.15.96:0/3434823364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 -- 172.21.15.96:0/3434823364 >> 172.21.15.96:0/3434823364 conn(0x559c2c9a58b0 msgr2=0x559c2c9a5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 -- 172.21.15.96:0/3434823364 shutdown_connections 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.697+0000 7f108140c740 1 -- 172.21.15.96:0/3434823364 wait complete. 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 -- start start 2024-03-18T15:41:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 Processor -- start 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c2c9b7200 con 0x559c2c9af140 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c2c8cbd70 con 0x559c2c9b42b0 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559c2c9b7380 con 0x559c2c9ae750 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f108140c740 10 client.0 _ll_register_callbacks cb 0x559c2c623a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f1077fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f10777fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55656/0 (socket says 172.21.15.96:55656) 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f10777fe640 1 -- 172.21.15.96:0/945859858 learned_addr learned my addr 172.21.15.96:0/945859858 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.698+0000 7f107cce5640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 -- 172.21.15.96:0/945859858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 msgr2=0x559c2c9bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 -- 172.21.15.96:0/945859858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 msgr2=0x559c2c9b8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 -- 172.21.15.96:0/945859858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2c624210 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f107cce5640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10777fe640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f1077fff640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9b8a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f106802c170 tx=0x7f1068009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459ccae6db27e821 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106804e070 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1068003cc0 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f106c09b4d0 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f106805a050 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f108140c740 10 client.25588 fetch_fsmap learned FSMap version 26 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c2c6f0620 con 0x559c2c9ae750 2024-03-18T15:41:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c09b4d0 con 0x559c2c9ae750 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f1068053440 con 0x559c2c9ae750 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f108140c740 10 client.25588 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.699+0000 7f108140c740 10 client.25588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f108140c740 20 client.25588 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2c9c4800 0x559c2c9c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f108140c740 4 client.25588 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c2c91cf80 con 0x559c2c9c4800 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f10777fe640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2c9c4800 0x559c2c9c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f10577fe640 10 client.25588 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.700+0000 7f10777fe640 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2c9c4800 0x559c2c9c6c70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ab8e41c3664fb7 server_cookie=282502b10b1406a4 in_seq=0 out_seq=0 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f10577fe640 1 -- 172.21.15.96:0/945859858 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559c2c91cf80 con 0x559c2c9c4800 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f10577fe640 10 client.25588 handle_command_reply: tid=0 2024-03-18T15:41:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 client.25588 shutdown 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 msgr2=0x559c2c9b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9b8a60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f106802c170 tx=0x7f1068009260 comp rx=0 tx=0).stop 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 shutdown_connections 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2c9c4800 0x559c2c9c6c70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9af140 0x559c2c9b8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f10577fe640 0 client.25588 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 --2- 172.21.15.96:0/945859858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9ae750 0x559c2c9b8a60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 >> 172.21.15.96:0/945859858 conn(0x559c2c9a58b0 msgr2=0x559c2c9a4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 shutdown_connections 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 1 -- 172.21.15.96:0/945859858 wait complete. 2024-03-18T15:41:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.701+0000 7f108140c740 20 client.25588 trim_cache size 0 max 16384 2024-03-18T15:41:06.716 INFO:tasks.workunit.client.0.smithi096.stdout:checking for inline... inline 2024-03-18T15:41:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 Processor -- start 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 -- start start 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 0x559c2c9bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9b42b0 0x559c2c9a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a6b90 con 0x559c2c9ae750 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a8700 con 0x559c2c9af140 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f108140c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a7260 con 0x559c2c9b42b0 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f1077fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f1077fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55662/0 (socket says 172.21.15.96:55662) 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f1077fff640 1 -- 172.21.15.96:0/1147224651 learned_addr learned my addr 172.21.15.96:0/1147224651 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f10777fe640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 0x559c2c9bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.716+0000 7f107cce5640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9b42b0 0x559c2c9a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 -- 172.21.15.96:0/1147224651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9b42b0 msgr2=0x559c2c9a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9b42b0 0x559c2c9a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 -- 172.21.15.96:0/1147224651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 msgr2=0x559c2c9b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 -- 172.21.15.96:0/1147224651 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c2c9af520 con 0x559c2c9af140 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f1077fff640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f10777fe640 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 0x559c2c9bff30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f106c097230 tx=0x7f106c0ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c56a881c947548ba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f105dffb640 1 -- 172.21.15.96:0/1147224651 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c098600 con 0x559c2c9af140 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f105dffb640 1 -- 172.21.15.96:0/1147224651 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c098420 con 0x559c2c9af140 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.717+0000 7f105dffb640 1 -- 172.21.15.96:0/1147224651 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c098820 con 0x559c2c9af140 2024-03-18T15:41:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- 172.21.15.96:0/1147224651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 msgr2=0x559c2c9bff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 0x559c2c9bff30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f106c097230 tx=0x7f106c0ce040 comp rx=0 tx=0).stop 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- 172.21.15.96:0/1147224651 shutdown_connections 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9b42b0 0x559c2c9a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9af140 0x559c2c9bff30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- 172.21.15.96:0/1147224651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- 172.21.15.96:0/1147224651 >> 172.21.15.96:0/1147224651 conn(0x559c2c9a58b0 msgr2=0x559c2c9af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- 172.21.15.96:0/1147224651 shutdown_connections 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- 172.21.15.96:0/1147224651 wait complete. 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- start start 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 Processor -- start 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9af140 0x559c2c9bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a8000 con 0x559c2c9ae750 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a7930 con 0x559c2c9b42b0 2024-03-18T15:41:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.718+0000 7f108140c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10781a4b70 con 0x559c2c9af140 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f108140c740 10 client.0 _ll_register_callbacks cb 0x7f1068041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55676/0 (socket says 172.21.15.96:55676) 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 -- 172.21.15.96:0/1106614683 learned_addr learned my addr 172.21.15.96:0/1106614683 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f10777fe640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9af140 0x559c2c9bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f107cce5640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 -- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9af140 msgr2=0x559c2c9bdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9af140 0x559c2c9bdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 -- 172.21.15.96:0/1106614683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 msgr2=0x559c2c9bebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 -- 172.21.15.96:0/1106614683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c2c9bf1c0 con 0x559c2c9ae750 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f1077fff640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1068053a20 tx=0x7f1068053670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a6a51b5eaa3a4a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f10577fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068063060 con 0x559c2c9ae750 2024-03-18T15:41:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1068003bc0 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f106c098420 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f106804e050 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f108140c740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.719+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c2ca2abc0 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c098420 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7f1068004990 con 0x559c2c9ae750 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 20 client.15753 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2ca2b590 0x559c2ca2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 4 client.15753 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c2c9172d0 con 0x559c2ca2b590 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f10777fe640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2ca2b590 0x559c2ca2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f10577fe640 10 client.15753 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.720+0000 7f10777fe640 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2ca2b590 0x559c2ca2d9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab78923043ab8ef3 server_cookie=70c23e03fde3b510 in_seq=0 out_seq=0 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.737+0000 7f10577fe640 1 -- 172.21.15.96:0/1106614683 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c2c9172d0 con 0x559c2ca2b590 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f10577fe640 10 client.15753 handle_command_reply: tid=0 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 client.15753 shutdown 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 msgr2=0x559c2c9bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1068053a20 tx=0x7f1068053670 comp rx=0 tx=0).stop 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 shutdown_connections 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c2c9b42b0 0x559c2c9bebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559c2c9af140 0x559c2c9bdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559c2ca2b590 0x559c2ca2d9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 --2- 172.21.15.96:0/1106614683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c2c9ae750 0x559c2c9bd680 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f10577fe640 0 client.15753 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 >> 172.21.15.96:0/1106614683 conn(0x559c2c9a58b0 msgr2=0x559c2c9b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 shutdown_connections 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 1 -- 172.21.15.96:0/1106614683 wait complete. 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.738+0000 7f108140c740 20 client.15753 trim_cache size 0 max 16384 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:41:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 msgr2=0x7f10600a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 0x7f10600a4e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1068000c00 tx=0x7f1068005fd0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 msgr2=0x7f1078197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00de10 comp rx=0 tx=0).stop 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 shutdown_connections 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f10600a2940 0x7f10600a4e00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10781022e0 0x7f107819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1078101920 0x7f1078197fd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 --2- 172.21.15.96:0/4152065392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1078100730 0x7f1078197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 >> 172.21.15.96:0/4152065392 conn(0x7f10780fbcc0 msgr2=0x7f10780fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 shutdown_connections 2024-03-18T15:41:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:06.739+0000 7f107e748640 1 -- 172.21.15.96:0/4152065392 wait complete. 2024-03-18T15:41:06.791 INFO:tasks.workunit.client.0.smithi096.stdout:checking for working alloca.h... yes 2024-03-18T15:41:06.843 INFO:tasks.workunit.client.0.smithi096.stdout:checking for alloca... yes 2024-03-18T15:41:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:06 smithi137 ceph-mon[35868]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 21 op/s 2024-03-18T15:41:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:06 smithi193 ceph-mon[36702]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 21 op/s 2024-03-18T15:41:06.918 INFO:tasks.workunit.client.0.smithi096.stdout:checking for working memcmp... yes 2024-03-18T15:41:06.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:06 smithi096 ceph-mon[31930]: pgmap v382: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 21 op/s 2024-03-18T15:41:06.983 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strftime... yes 2024-03-18T15:41:07.069 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-03-18T15:41:07.139 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether stat accepts an empty string... no 2024-03-18T15:41:07.203 INFO:tasks.workunit.client.0.smithi096.stdout:checking for vprintf... yes 2024-03-18T15:41:07.268 INFO:tasks.workunit.client.0.smithi096.stdout:checking for _doprnt... no 2024-03-18T15:41:07.291 INFO:tasks.workunit.client.0.smithi096.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-03-18T15:41:07.302 INFO:tasks.workunit.client.0.smithi096.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-03-18T15:41:07.366 INFO:tasks.workunit.client.0.smithi096.stdout:checking for the pthreads library -lpthreads... no 2024-03-18T15:41:07.425 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether pthreads work without any flags... yes 2024-03-18T15:41:07.472 INFO:tasks.workunit.client.0.smithi096.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-03-18T15:41:07.472 INFO:tasks.workunit.client.0.smithi096.stdout:checking if more special flags are required for pthreads... no 2024-03-18T15:41:07.472 INFO:tasks.workunit.client.0.smithi096.stdout:checking for cc_r... gcc 2024-03-18T15:41:07.553 INFO:tasks.workunit.client.0.smithi096.stdout:checking for setrlimit... yes 2024-03-18T15:41:07.620 INFO:tasks.workunit.client.0.smithi096.stdout:checking for setproctitle... no 2024-03-18T15:41:07.688 INFO:tasks.workunit.client.0.smithi096.stdout:checking for getopt_long... yes 2024-03-18T15:41:07.756 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strtoull... yes 2024-03-18T15:41:07.823 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strtoq... yes 2024-03-18T15:41:07.894 INFO:tasks.workunit.client.0.smithi096.stdout:checking for snprintf... yes 2024-03-18T15:41:07.963 INFO:tasks.workunit.client.0.smithi096.stdout:checking for vsnprintf... yes 2024-03-18T15:41:08.031 INFO:tasks.workunit.client.0.smithi096.stdout:checking for vfprintf... yes 2024-03-18T15:41:08.094 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gethostname... yes 2024-03-18T15:41:08.157 INFO:tasks.workunit.client.0.smithi096.stdout:checking for setlocale... yes 2024-03-18T15:41:08.221 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sync... yes 2024-03-18T15:41:08.294 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether snprintf is C99 conformant... done 2024-03-18T15:41:08.307 INFO:tasks.workunit.client.0.smithi096.stdout:configure: creating ./config.status 2024-03-18T15:41:08.497 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating Makefile 2024-03-18T15:41:08.515 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating src/Makefile 2024-03-18T15:41:08.535 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating man/Makefile 2024-03-18T15:41:08.547 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating config.h 2024-03-18T15:41:08.611 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: executing depfiles commands 2024-03-18T15:41:08.754 INFO:tasks.workunit.client.0.smithi096.stderr:+ make 2024-03-18T15:41:08.764 INFO:tasks.workunit.client.0.smithi096.stdout:make all-recursive 2024-03-18T15:41:08.766 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-03-18T15:41:08.769 INFO:tasks.workunit.client.0.smithi096.stdout:Making all in src 2024-03-18T15:41:08.775 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-03-18T15:41:08.776 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-03-18T15:41:08.776 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-03-18T15:41:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:08 smithi137 ceph-mon[35868]: pgmap v383: 97 pgs: 97 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-03-18T15:41:08.864 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-03-18T15:41:08.864 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-03-18T15:41:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:08 smithi193 ceph-mon[36702]: pgmap v383: 97 pgs: 97 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-03-18T15:41:08.929 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-03-18T15:41:08.929 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-03-18T15:41:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:08 smithi096 ceph-mon[31930]: pgmap v383: 97 pgs: 97 active+clean; 1.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-03-18T15:41:08.982 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-03-18T15:41:08.983 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-03-18T15:41:09.033 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-03-18T15:41:09.033 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-03-18T15:41:09.081 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-03-18T15:41:09.081 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-03-18T15:41:09.126 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-03-18T15:41:09.127 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-03-18T15:41:09.202 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-03-18T15:41:09.202 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-03-18T15:41:09.242 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-03-18T15:41:09.242 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-03-18T15:41:09.281 INFO:tasks.workunit.client.0.smithi096.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-03-18T15:41:09.281 INFO:tasks.workunit.client.0.smithi096.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-03-18T15:41:09.317 INFO:tasks.workunit.client.0.smithi096.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-03-18T15:41:09.335 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-03-18T15:41:09.335 INFO:tasks.workunit.client.0.smithi096.stdout:Making all in man 2024-03-18T15:41:09.374 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-03-18T15:41:09.374 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Nothing to be done for 'all'. 2024-03-18T15:41:09.374 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-03-18T15:41:09.375 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-03-18T15:41:09.376 INFO:tasks.workunit.client.0.smithi096.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-03-18T15:41:09.377 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-03-18T15:41:09.377 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd src 2024-03-18T15:41:09.377 INFO:tasks.workunit.client.0.smithi096.stderr:+ mkdir blogtest_in 2024-03-18T15:41:09.379 INFO:tasks.workunit.client.0.smithi096.stdout:running blogbench 2024-03-18T15:41:09.380 INFO:tasks.workunit.client.0.smithi096.stderr:+ echo 'running blogbench' 2024-03-18T15:41:09.380 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./blogbench -d blogtest_in 2024-03-18T15:41:09.380 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-18T15:41:09.380 INFO:tasks.workunit.client.0.smithi096.stdout:Frequency = 10 secs 2024-03-18T15:41:09.380 INFO:tasks.workunit.client.0.smithi096.stdout:Scratch dir = [blogtest_in] 2024-03-18T15:41:10.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:10 smithi137 ceph-mon[35868]: pgmap v384: 97 pgs: 97 active+clean; 3.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 287 KiB/s wr, 33 op/s 2024-03-18T15:41:10.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:10 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:10 smithi193 ceph-mon[36702]: pgmap v384: 97 pgs: 97 active+clean; 3.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 287 KiB/s wr, 33 op/s 2024-03-18T15:41:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:10 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:10 smithi096 ceph-mon[31930]: pgmap v384: 97 pgs: 97 active+clean; 3.8 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 287 KiB/s wr, 33 op/s 2024-03-18T15:41:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:10 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:11.325 INFO:tasks.workunit.client.0.smithi096.stdout:Spawning 3 writers... 2024-03-18T15:41:11.325 INFO:tasks.workunit.client.0.smithi096.stdout:Spawning 1 rewriters... 2024-03-18T15:41:11.325 INFO:tasks.workunit.client.0.smithi096.stdout:Spawning 5 commenters... 2024-03-18T15:41:11.325 INFO:tasks.workunit.client.0.smithi096.stdout:Spawning 100 readers... 2024-03-18T15:41:11.327 INFO:tasks.workunit.client.0.smithi096.stdout:Benchmarking for 30 iterations. 2024-03-18T15:41:11.327 INFO:tasks.workunit.client.0.smithi096.stdout:The test will run during 5 minutes. 2024-03-18T15:41:11.327 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-18T15:41:11.327 INFO:tasks.workunit.client.0.smithi096.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-03-18T15:41:12.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:12 smithi137 ceph-mon[35868]: pgmap v385: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 312 KiB/s wr, 38 op/s 2024-03-18T15:41:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:12 smithi193 ceph-mon[36702]: pgmap v385: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 312 KiB/s wr, 38 op/s 2024-03-18T15:41:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:12 smithi096 ceph-mon[31930]: pgmap v385: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 312 KiB/s wr, 38 op/s 2024-03-18T15:41:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:14 smithi137 ceph-mon[35868]: pgmap v386: 97 pgs: 97 active+clean; 7.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 532 KiB/s wr, 48 op/s 2024-03-18T15:41:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:14 smithi193 ceph-mon[36702]: pgmap v386: 97 pgs: 97 active+clean; 7.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 532 KiB/s wr, 48 op/s 2024-03-18T15:41:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:14 smithi096 ceph-mon[31930]: pgmap v386: 97 pgs: 97 active+clean; 7.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 532 KiB/s wr, 48 op/s 2024-03-18T15:41:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:16 smithi193 ceph-mon[36702]: pgmap v387: 97 pgs: 97 active+clean; 7.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 543 KiB/s wr, 52 op/s 2024-03-18T15:41:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:16 smithi096 ceph-mon[31930]: pgmap v387: 97 pgs: 97 active+clean; 7.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 543 KiB/s wr, 52 op/s 2024-03-18T15:41:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:16 smithi137 ceph-mon[35868]: pgmap v387: 97 pgs: 97 active+clean; 7.2 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 543 KiB/s wr, 52 op/s 2024-03-18T15:41:17.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:41:17.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:41:17.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 -- 172.21.15.96:0/3468445721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 msgr2=0x7f224c1024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 --2- 172.21.15.96:0/3468445721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c1024a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2248009900 tx=0x7f224802bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:17.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 -- 172.21.15.96:0/3468445721 shutdown_connections 2024-03-18T15:41:17.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 --2- 172.21.15.96:0/3468445721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c105390 0x7f224c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 --2- 172.21.15.96:0/3468445721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c1029e0 0x7f224c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 --2- 172.21.15.96:0/3468445721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c1024a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 -- 172.21.15.96:0/3468445721 >> 172.21.15.96:0/3468445721 conn(0x7f224c0fbc90 msgr2=0x7f224c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 -- 172.21.15.96:0/3468445721 shutdown_connections 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.433+0000 7f22539c6640 1 -- 172.21.15.96:0/3468445721 wait complete. 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 Processor -- start 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 -- start start 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1029e0 0x7f224c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c105390 0x7f224c1a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f224c113a50 con 0x7f224c100070 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1138d0 con 0x7f224c1029e0 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f22539c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f224c113bd0 con 0x7f224c105390 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52824/0 (socket says 172.21.15.96:52824) 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 -- 172.21.15.96:0/2717368595 learned_addr learned my addr 172.21.15.96:0/2717368595 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251f63640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c105390 0x7f224c1a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2250f61640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1029e0 0x7f224c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 -- 172.21.15.96:0/2717368595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c105390 msgr2=0x7f224c1a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c105390 0x7f224c1a56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 -- 172.21.15.96:0/2717368595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1029e0 msgr2=0x7f224c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1029e0 0x7f224c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 -- 172.21.15.96:0/2717368595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1a5d60 con 0x7f224c100070 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.434+0000 7f2251762640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f2248005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.435+0000 7f22427fc640 1 -- 172.21.15.96:0/2717368595 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248041070 con 0x7f224c100070 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.435+0000 7f22427fc640 1 -- 172.21.15.96:0/2717368595 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480044d0 con 0x7f224c100070 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.435+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1a5ff0 con 0x7f224c100070 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.435+0000 7f22427fc640 1 -- 172.21.15.96:0/2717368595 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248034440 con 0x7f224c100070 2024-03-18T15:41:17.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.435+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1a6450 con 0x7f224c100070 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.437+0000 7f22427fc640 1 -- 172.21.15.96:0/2717368595 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f2248044070 con 0x7f224c100070 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.437+0000 7f22565b2740 1 Processor -- start 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.437+0000 7f22565b2740 1 -- start start 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.437+0000 7f22427fc640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 0x7f22380a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702388590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702390160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567022ed500 con 0x556702388b60 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556702295be0 con 0x556702388170 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22565b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567022e95a0 con 0x55670238dcd0 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251f63640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 0x7f22380a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22427fc640 1 -- 172.21.15.96:0/2717368595 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f22480f0690 con 0x7f224c100070 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2250f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702388590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2250f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52834/0 (socket says 172.21.15.96:52834) 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2250f61640 1 -- 172.21.15.96:0/1610236506 learned_addr learned my addr 172.21.15.96:0/1610236506 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251f63640 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702390160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 -- 172.21.15.96:0/1610236506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 msgr2=0x556702390160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702390160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 -- 172.21.15.96:0/1610236506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 msgr2=0x55670238d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 -- 172.21.15.96:0/1610236506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567022a94b0 con 0x556702388170 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2250f61640 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251f63640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 0x7f22380a4dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2244005fd0 tx=0x7f2244009e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f2251762640 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702388590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f22480ee120 tx=0x7f22480b93c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ec34839f2930ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22377fe640 1 -- 172.21.15.96:0/1610236506 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224804b040 con 0x556702388170 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22377fe640 1 -- 172.21.15.96:0/1610236506 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480bd060 con 0x556702388170 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.438+0000 7f22377fe640 1 -- 172.21.15.96:0/1610236506 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480de2a0 con 0x556702388170 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- 172.21.15.96:0/1610236506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 msgr2=0x556702388590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702388590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f22480ee120 tx=0x7f22480b93c0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- 172.21.15.96:0/1610236506 shutdown_connections 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702390160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x55670238d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 --2- 172.21.15.96:0/1610236506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702388590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- 172.21.15.96:0/1610236506 >> 172.21.15.96:0/1610236506 conn(0x5567023836e0 msgr2=0x556702383ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- 172.21.15.96:0/1610236506 shutdown_connections 2024-03-18T15:41:17.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- 172.21.15.96:0/1610236506 wait complete. 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 -- start start 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.439+0000 7f22565b2740 1 Processor -- start 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702392390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388b60 0x5567023928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556702390a40 con 0x55670238dcd0 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5567022a9160 con 0x556702388170 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556702390bc0 con 0x556702388b60 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f22565b2740 10 client.0 _ll_register_callbacks cb 0x55670228d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251762640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702392390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52840/0 (socket says 172.21.15.96:52840) 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2250f61640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388b60 0x5567023928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 -- 172.21.15.96:0/2222248676 learned_addr learned my addr 172.21.15.96:0/2222248676 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 -- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388b60 msgr2=0x5567023928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388b60 0x5567023928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 -- 172.21.15.96:0/2222248676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 msgr2=0x556702392390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702392390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251f63640 1 -- 172.21.15.96:0/2222248676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567020cb7d0 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.440+0000 7f2251762640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702392390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2251f63640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2244009270 tx=0x7f2244004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def6bd67b6426d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244003970 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2244003b10 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22480bd060 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2244031030 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567020013f0 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480bd060 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f2244020050 con 0x55670238dcd0 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:17.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 20 client.15765 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55670239ded0 0x5567023a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 4 client.15765 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567022faa40 con 0x55670239ded0 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.441+0000 7f2251762640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55670239ded0 0x5567023a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.442+0000 7f2234ff9640 10 client.15765 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.442+0000 7f2251762640 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55670239ded0 0x5567023a0350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efe08b9b3e677ae server_cookie=b24ba1bf3da483ef in_seq=0 out_seq=0 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f2234ff9640 1 -- 172.21.15.96:0/2222248676 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5567022faa40 con 0x55670239ded0 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f2234ff9640 10 client.15765 handle_command_reply: tid=0 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 client.15765 shutdown 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 msgr2=0x556702399910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2244009270 tx=0x7f2244004ab0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 shutdown_connections 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55670239ded0 0x5567023a0350 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670238dcd0 0x556702399910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388b60 0x5567023928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f2234ff9640 0 client.15765 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 --2- 172.21.15.96:0/2222248676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388170 0x556702392390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 >> 172.21.15.96:0/2222248676 conn(0x5567023836e0 msgr2=0x556702382900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 shutdown_connections 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 1 -- 172.21.15.96:0/2222248676 wait complete. 2024-03-18T15:41:17.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.443+0000 7f22565b2740 20 client.15765 trim_cache size 0 max 16384 2024-03-18T15:41:17.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 Processor -- start 2024-03-18T15:41:17.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 -- start start 2024-03-18T15:41:17.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388b60 0x5567023998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702382070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1af2a0 con 0x556702388170 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b0ea0 con 0x556702388b60 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f22565b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1af970 con 0x55670238dcd0 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f2251762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f2251762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52850/0 (socket says 172.21.15.96:52850) 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f2251762640 1 -- 172.21.15.96:0/3349322950 learned_addr learned my addr 172.21.15.96:0/3349322950 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f2250f61640 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388b60 0x5567023998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.460+0000 7f2251f63640 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702382070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 -- 172.21.15.96:0/3349322950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 msgr2=0x556702382070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702382070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 -- 172.21.15.96:0/3349322950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388b60 msgr2=0x5567023998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388b60 0x5567023998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 -- 172.21.15.96:0/3349322950 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556702391490 con 0x556702388170 2024-03-18T15:41:17.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2251762640 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22480ee590 tx=0x7f2248000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401301923f4d7286 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2236ffd640 1 -- 172.21.15.96:0/3349322950 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224804b040 con 0x556702388170 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2236ffd640 1 -- 172.21.15.96:0/3349322950 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480bd060 con 0x556702388170 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f2236ffd640 1 -- 172.21.15.96:0/3349322950 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480b96d0 con 0x556702388170 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f22565b2740 1 -- 172.21.15.96:0/3349322950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 msgr2=0x55670237e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.461+0000 7f22565b2740 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22480ee590 tx=0x7f2248000bc0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- 172.21.15.96:0/3349322950 shutdown_connections 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55670238dcd0 0x556702382070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556702388b60 0x5567023998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- 172.21.15.96:0/3349322950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388170 0x55670237e8e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- 172.21.15.96:0/3349322950 >> 172.21.15.96:0/3349322950 conn(0x5567023836e0 msgr2=0x556702389600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- 172.21.15.96:0/3349322950 shutdown_connections 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- 172.21.15.96:0/3349322950 wait complete. 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- start start 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 Processor -- start 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388170 0x556702396a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55670238dcd0 0x556702397fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b0710 con 0x556702388b60 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1b0040 con 0x55670238dcd0 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1ad280 con 0x556702388170 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f22565b2740 10 client.0 _ll_register_callbacks cb 0x7f22440135a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f2250f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f2250f61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52866/0 (socket says 172.21.15.96:52866) 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f2251762640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388170 0x556702396a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f2250f61640 1 -- 172.21.15.96:0/555679691 learned_addr learned my addr 172.21.15.96:0/555679691 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.462+0000 7f2251f63640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55670238dcd0 0x556702397fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 -- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388170 msgr2=0x556702396a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388170 0x556702396a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 -- 172.21.15.96:0/555679691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55670238dcd0 msgr2=0x556702397fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55670238dcd0 0x556702397fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 -- 172.21.15.96:0/555679691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556701fc10f0 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2250f61640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5567023895d0 tx=0x7f223c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61538bffe2cbe26e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c021020 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f223c005220 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22480bd060 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f223c005420 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f22565b2740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567022b49d0 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22480bd060 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f223c011950 con 0x556702388b60 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.463+0000 7f22565b2740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 20 client.15771 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556702404600 0x556702406a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 4 client.15771 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567022f4d90 con 0x556702404600 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f2251762640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556702404600 0x556702406a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f2234ff9640 10 client.15771 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f2251762640 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556702404600 0x556702406a20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8acd30e49f576978 server_cookie=2c448aeedee926cb in_seq=0 out_seq=0 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f2234ff9640 1 -- 172.21.15.96:0/555679691 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5567022f4d90 con 0x556702404600 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f2234ff9640 10 client.15771 handle_command_reply: tid=0 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 client.15771 shutdown 2024-03-18T15:41:17.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 msgr2=0x556702396fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5567023895d0 tx=0x7f223c0039c0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 shutdown_connections 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55670238dcd0 0x556702397fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556702404600 0x556702406a20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556702388b60 0x556702396fd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f2234ff9640 0 client.15771 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.464+0000 7f22565b2740 1 --2- 172.21.15.96:0/555679691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556702388170 0x556702396a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 >> 172.21.15.96:0/555679691 conn(0x5567023836e0 msgr2=0x55670237e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 shutdown_connections 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22565b2740 1 -- 172.21.15.96:0/555679691 wait complete. 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22565b2740 20 client.15771 trim_cache size 0 max 16384 2024-03-18T15:41:17.472 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:17.473 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:41:17.473 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:41:17.473 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:17.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 msgr2=0x7f22380a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 0x7f22380a4dd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2244005fd0 tx=0x7f2244009e60 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 msgr2=0x7f224c19e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f2248005d50 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 shutdown_connections 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f22380a2910 0x7f22380a4dd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f224c105390 0x7f224c1a56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1029e0 0x7f224c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 --2- 172.21.15.96:0/2717368595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f224c100070 0x7f224c19e190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 >> 172.21.15.96:0/2717368595 conn(0x7f224c0fbc90 msgr2=0x7f224c1032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 shutdown_connections 2024-03-18T15:41:17.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.465+0000 7f22539c6640 1 -- 172.21.15.96:0/2717368595 wait complete. 2024-03-18T15:41:17.476 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:41:17.476 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 -- 172.21.15.96:0/3057106231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e01022d0 msgr2=0x7f34e0108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 --2- 172.21.15.96:0/3057106231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e01022d0 0x7f34e0108b60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 -- 172.21.15.96:0/3057106231 shutdown_connections 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 --2- 172.21.15.96:0/3057106231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e01022d0 0x7f34e0108b60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 --2- 172.21.15.96:0/3057106231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e0101910 0x7f34e0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 --2- 172.21.15.96:0/3057106231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0100720 0x7f34e0100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 -- 172.21.15.96:0/3057106231 >> 172.21.15.96:0/3057106231 conn(0x7f34e00fbc90 msgr2=0x7f34e00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 -- 172.21.15.96:0/3057106231 shutdown_connections 2024-03-18T15:41:17.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.577+0000 7f34e4861640 1 -- 172.21.15.96:0/3057106231 wait complete. 2024-03-18T15:41:17.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 Processor -- start 2024-03-18T15:41:17.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 -- start start 2024-03-18T15:41:17.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0101910 0x7f34e019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e01022d0 0x7f34e01a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34e010c3a0 con 0x7f34e0100720 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34e010c220 con 0x7f34e0101910 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f34e010c520 con 0x7f34e01022d0 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52886/0 (socket says 172.21.15.96:52886) 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 -- 172.21.15.96:0/3662929671 learned_addr learned my addr 172.21.15.96:0/3662929671 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34ded9d640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e01022d0 0x7f34e01a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34ddd9b640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0101910 0x7f34e019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 -- 172.21.15.96:0/3662929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e01022d0 msgr2=0x7f34e01a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e01022d0 0x7f34e01a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 -- 172.21.15.96:0/3662929671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0101910 msgr2=0x7f34e019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0101910 0x7f34e019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 -- 172.21.15.96:0/3662929671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a1990 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34de59c640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009d20 tx=0x7f34d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8010630 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d8010cb0 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a1c20 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.578+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8018de0 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.579+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a2170 con 0x7f34e0100720 2024-03-18T15:41:17.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.580+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e01079e0 con 0x7f34e0100720 2024-03-18T15:41:17.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.580+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f34d801f020 con 0x7f34e0100720 2024-03-18T15:41:17.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.581+0000 7f34d77fe640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 0x7f34cc0a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:17.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.581+0000 7f34ddd9b640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 0x7f34cc0a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:17.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.581+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f34d80cf060 con 0x7f34e0100720 2024-03-18T15:41:17.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.582+0000 7f34ddd9b640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 0x7f34cc0a4d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f34e006a750 tx=0x7f34c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:17.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.586+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f34d809bf70 con 0x7f34e0100720 2024-03-18T15:41:17.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.896+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34e00623e0 con 0x7f34e0100720 2024-03-18T15:41:17.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.897+0000 7f34d77fe640 1 -- 172.21.15.96:0/3662929671 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f34d809b870 con 0x7f34e0100720 2024-03-18T15:41:17.897 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:41:17.898 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:41:09.437656+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24472,24493,24485,24452,24460]},"id":1}]} 2024-03-18T15:41:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 msgr2=0x7f34cc0a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 0x7f34cc0a4d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f34e006a750 tx=0x7f34c800b390 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 msgr2=0x7f34e006a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009d20 tx=0x7f34d800c6a0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 shutdown_connections 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f34cc0a28c0 0x7f34cc0a4d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34e01022d0 0x7f34e01a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e0101910 0x7f34e019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 --2- 172.21.15.96:0/3662929671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e0100720 0x7f34e006a9e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 >> 172.21.15.96:0/3662929671 conn(0x7f34e00fbc90 msgr2=0x7f34e0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 shutdown_connections 2024-03-18T15:41:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:17.899+0000 7f34e4861640 1 -- 172.21.15.96:0/3662929671 wait complete. 2024-03-18T15:41:17.912 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4205006746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b81034f0 msgr2=0x7f33b8109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4205006746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b81034f0 0x7f33b8109d80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009ca0 tx=0x7f33b002c140 comp rx=0 tx=0).stop 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4205006746 shutdown_connections 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4205006746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b81034f0 0x7f33b8109d80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4205006746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b8102b30 0x7f33b8102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4205006746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b80ffac0 0x7f33b80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4205006746 >> 172.21.15.96:0/4205006746 conn(0x7f33b80fb410 msgr2=0x7f33b80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4205006746 shutdown_connections 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4205006746 wait complete. 2024-03-18T15:41:18.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.017+0000 7f33bdd3b640 1 Processor -- start 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 -- start start 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b8102b30 0x7f33b8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b81034f0 0x7f33b819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33b810d490 con 0x7f33b80ffac0 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b810d310 con 0x7f33b81034f0 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33bdd3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33b810d610 con 0x7f33b8102b30 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52898/0 (socket says 172.21.15.96:52898) 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 -- 172.21.15.96:0/4236444743 learned_addr learned my addr 172.21.15.96:0/4236444743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b8102b30 0x7f33b8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b7fff640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b81034f0 0x7f33b819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 -- 172.21.15.96:0/4236444743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b8102b30 msgr2=0x7f33b8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b8102b30 0x7f33b8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 -- 172.21.15.96:0/4236444743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b81034f0 msgr2=0x7f33b819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b81034f0 0x7f33b819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 -- 172.21.15.96:0/4236444743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b819f6c0 con 0x7f33b80ffac0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.018+0000 7f33b77fe640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33a800e9c0 tx=0x7f33a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.019+0000 7f33b4ff9640 1 -- 172.21.15.96:0/4236444743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a800cd80 con 0x7f33b80ffac0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.019+0000 7f33b4ff9640 1 -- 172.21.15.96:0/4236444743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a80035f0 con 0x7f33b80ffac0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.019+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b819f9b0 con 0x7f33b80ffac0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.019+0000 7f33b4ff9640 1 -- 172.21.15.96:0/4236444743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8010690 con 0x7f33b80ffac0 2024-03-18T15:41:18.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.019+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b819feb0 con 0x7f33b80ffac0 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33b4ff9640 1 -- 172.21.15.96:0/4236444743 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f33a80037a0 con 0x7f33b80ffac0 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 Processor -- start 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 -- start start 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a6850 0x558a320a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a3200b490 con 0x558a320ac3b0 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a31fb3c90 con 0x558a320a7240 2024-03-18T15:41:18.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.021+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a32007530 con 0x558a320a6850 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b4ff9640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 0x7f33a00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52902/0 (socket says 172.21.15.96:52902) 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3693747316 learned_addr learned my addr 172.21.15.96:0/3693747316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b7fff640 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a6850 0x558a320a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b7fff640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 0x7f33a00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b4ff9640 1 -- 172.21.15.96:0/4236444743 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f33a80cef80 con 0x7f33b80ffac0 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3693747316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a6850 msgr2=0x558a320a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a6850 0x558a320a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3693747316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 msgr2=0x558a320abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3693747316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3206e500 con 0x558a320ac3b0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b7fff640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 0x7f33a00a4d50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005cc0 tx=0x7f33b0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bc90 tx=0x7f33ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412037294f8b3640 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f339dffb640 1 -- 172.21.15.96:0/3693747316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac005220 con 0x558a320ac3b0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f339dffb640 1 -- 172.21.15.96:0/3693747316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ac00be40 con 0x558a320ac3b0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.022+0000 7f339dffb640 1 -- 172.21.15.96:0/3693747316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0117c0 con 0x558a320ac3b0 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3693747316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 msgr2=0x558a320ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bc90 tx=0x7f33ac0107a0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3693747316 shutdown_connections 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320ae840 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3693747316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a6850 0x558a320a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3693747316 >> 172.21.15.96:0/3693747316 conn(0x558a320a1dc0 msgr2=0x558a320a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3693747316 shutdown_connections 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3693747316 wait complete. 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.023+0000 7f33c0a0a740 1 -- start start 2024-03-18T15:41:18.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 Processor -- start 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558a320af360 con 0x558a320a6850 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558a31fc7230 con 0x558a320a7240 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558a320af4e0 con 0x558a320ac3b0 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 10 client.0 _ll_register_callbacks cb 0x558a32003010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52906/0 (socket says 172.21.15.96:52906) 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 -- 172.21.15.96:0/1173392038 learned_addr learned my addr 172.21.15.96:0/1173392038 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b7fff640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 -- 172.21.15.96:0/1173392038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 msgr2=0x558a320b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 -- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 msgr2=0x558a320b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 -- 172.21.15.96:0/1173392038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a31fd6100 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33b77fe640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33a809b490 tx=0x7f33a809b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5a69666c12ab4f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f3396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a809b790 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f33a8086070 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.024+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a320189d0 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33a809f8e0 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a31d1f3f0 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558a320189d0 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f33a80afb70 con 0x558a320a6850 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 10 client.15789 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 20 client.15789 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a320bcc00 0x558a320bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 4 client.15789 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a32012d20 con 0x558a320bcc00 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.025+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a320bcc00 0x558a320bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.026+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a320bcc00 0x558a320bf090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7946d6c04e9b54f server_cookie=ca664f0cdc09e0dc in_seq=0 out_seq=0 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.026+0000 7f3396ffd640 10 client.15789 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f3396ffd640 1 -- 172.21.15.96:0/1173392038 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558a32012d20 con 0x558a320bcc00 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f3396ffd640 10 client.15789 handle_command_reply: tid=0 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 client.15789 shutdown 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 msgr2=0x558a320b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f33a809b490 tx=0x7f33a809b2d0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 shutdown_connections 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a320bcc00 0x558a320bf090 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a7240 0x558a320b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/1173392038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a6850 0x558a320b0d60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f3396ffd640 0 client.15789 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 >> 172.21.15.96:0/1173392038 conn(0x558a320a1dc0 msgr2=0x558a320a0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 shutdown_connections 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 1 -- 172.21.15.96:0/1173392038 wait complete. 2024-03-18T15:41:18.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.027+0000 7f33c0a0a740 20 client.15789 trim_cache size 0 max 16384 2024-03-18T15:41:18.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.041+0000 7f33c0a0a740 1 Processor -- start 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.041+0000 7f33c0a0a740 1 -- start start 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a7240 0x558a320b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81a8d30 con 0x558a320ac3b0 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81aa930 con 0x558a320a6850 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81a9400 con 0x558a320a7240 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52914/0 (socket says 172.21.15.96:52914) 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b7fff640 1 -- 172.21.15.96:0/335114964 learned_addr learned my addr 172.21.15.96:0/335114964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a7240 0x558a320b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 -- 172.21.15.96:0/335114964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a7240 msgr2=0x558a320b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a7240 0x558a320b8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 -- 172.21.15.96:0/335114964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 msgr2=0x558a320c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 -- 172.21.15.96:0/335114964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a3209d640 con 0x558a320a6850 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b7fff640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f33b77fe640 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33a80afee0 tx=0x7f33a809b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1f6df0ca5a8867 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.042+0000 7f339d7fa640 1 -- 172.21.15.96:0/335114964 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a809b790 con 0x558a320a6850 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f339d7fa640 1 -- 172.21.15.96:0/335114964 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a80a3f30 con 0x558a320a6850 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f339d7fa640 1 -- 172.21.15.96:0/335114964 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80d1040 con 0x558a320a6850 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 -- 172.21.15.96:0/335114964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 msgr2=0x558a320a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33a80afee0 tx=0x7f33a809b400 comp rx=0 tx=0).stop 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 -- 172.21.15.96:0/335114964 shutdown_connections 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320ac3b0 0x558a320c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320a7240 0x558a320b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/335114964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320a5730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 -- 172.21.15.96:0/335114964 >> 172.21.15.96:0/335114964 conn(0x558a320a1dc0 msgr2=0x558a320a5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 -- 172.21.15.96:0/335114964 shutdown_connections 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.043+0000 7f33c0a0a740 1 -- 172.21.15.96:0/335114964 wait complete. 2024-03-18T15:41:18.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 -- start start 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 Processor -- start 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 0x558a320b5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81aa1a0 con 0x558a320a7240 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81a9ad0 con 0x558a320a6850 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33b81a6d10 con 0x558a320ac3b0 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33c0a0a740 10 client.0 _ll_register_callbacks cb 0x7f33a8003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 0x558a320b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33b77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60228/0 (socket says 172.21.15.96:60228) 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33b7fff640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.044+0000 7f33b77fe640 1 -- 172.21.15.96:0/3843015930 learned_addr learned my addr 172.21.15.96:0/3843015930 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3843015930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 msgr2=0x558a320b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 msgr2=0x558a320b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 -- 172.21.15.96:0/3843015930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a31de97d0 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b77fe640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b7fff640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33b6ffd640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 0x558a320b5950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bc60 tx=0x7f33ac005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37db13ee40878b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac02f060 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f33ac029630 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a80a3f30 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33ac005040 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33c0a0a740 10 client.15795 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a320b8030 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33a80a3f30 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f33ac011cd0 con 0x558a320a7240 2024-03-18T15:41:18.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33c0a0a740 10 client.15795 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.045+0000 7f33c0a0a740 10 client.15795 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 20 client.15795 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a32122f60 0x558a32125350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 4 client.15795 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a31f906a0 con 0x558a32122f60 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33b77fe640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a32122f60 0x558a32125350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f3396ffd640 10 client.15795 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:18.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33b77fe640 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a32122f60 0x558a32125350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=408a73cd0148a90f server_cookie=2ca64924ff7fc4cb in_seq=0 out_seq=0 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f3396ffd640 1 -- 172.21.15.96:0/3843015930 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a31f906a0 con 0x558a32122f60 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f3396ffd640 10 client.15795 handle_command_reply: tid=0 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 1 client.15795 shutdown 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 msgr2=0x558a320b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.046+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 0x558a320b5950 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ac00bc60 tx=0x7f33ac005a70 comp rx=0 tx=0).stop 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 shutdown_connections 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x558a32122f60 0x558a32125350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558a320ac3b0 0x558a320b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558a320a7240 0x558a320b5950 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f3396ffd640 0 client.15795 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 --2- 172.21.15.96:0/3843015930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558a320a6850 0x558a320b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 >> 172.21.15.96:0/3843015930 conn(0x558a320a1dc0 msgr2=0x558a320a58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 shutdown_connections 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 1 -- 172.21.15.96:0/3843015930 wait complete. 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33c0a0a740 20 client.15795 trim_cache size 0 max 16384 2024-03-18T15:41:18.053 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 msgr2=0x7f33a00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 0x7f33a00a4d50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005cc0 tx=0x7f33b0005c50 comp rx=0 tx=0).stop 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 msgr2=0x7f33b8197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33a800e9c0 tx=0x7f33a800ee80 comp rx=0 tx=0).stop 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 shutdown_connections 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33a00a2890 0x7f33a00a4d50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33b81034f0 0x7f33b819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b8102b30 0x7f33b8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 --2- 172.21.15.96:0/4236444743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33b80ffac0 0x7f33b8197aa0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 >> 172.21.15.96:0/4236444743 conn(0x7f33b80fb410 msgr2=0x7f33b8105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 shutdown_connections 2024-03-18T15:41:18.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.047+0000 7f33bdd3b640 1 -- 172.21.15.96:0/4236444743 wait complete. 2024-03-18T15:41:18.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:41:18.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- 172.21.15.96:0/1985283292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 msgr2=0x7f09a8109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 --2- 172.21.15.96:0/1985283292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a8109c30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009ca0 tx=0x7f09a002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- 172.21.15.96:0/1985283292 shutdown_connections 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 --2- 172.21.15.96:0/1985283292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a8109c30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 --2- 172.21.15.96:0/1985283292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81029e0 0x7f09a8102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 --2- 172.21.15.96:0/1985283292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81017f0 0x7f09a8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- 172.21.15.96:0/1985283292 >> 172.21.15.96:0/1985283292 conn(0x7f09a80fcd80 msgr2=0x7f09a80ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- 172.21.15.96:0/1985283292 shutdown_connections 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- 172.21.15.96:0/1985283292 wait complete. 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 Processor -- start 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.165+0000 7f09af3d4640 1 -- start start 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81017f0 0x7f09a8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81029e0 0x7f09a8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810d470 con 0x7f09a81033a0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810d2f0 con 0x7f09a81029e0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a810d5f0 con 0x7f09a81017f0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52948/0 (socket says 172.21.15.96:52948) 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 -- 172.21.15.96:0/3222590304 learned_addr learned my addr 172.21.15.96:0/3222590304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad170640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81017f0 0x7f09a8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ac96f640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81029e0 0x7f09a8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 -- 172.21.15.96:0/3222590304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81017f0 msgr2=0x7f09a8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81017f0 0x7f09a8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 -- 172.21.15.96:0/3222590304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81029e0 msgr2=0x7f09a8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81029e0 0x7f09a8197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 -- 172.21.15.96:0/3222590304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a819f6d0 con 0x7f09a81033a0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09ad971640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09a0037040 tx=0x7f09a0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f099e7fc640 1 -- 172.21.15.96:0/3222590304 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0041020 con 0x7f09a81033a0 2024-03-18T15:41:18.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f099e7fc640 1 -- 172.21.15.96:0/3222590304 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0034cd0 con 0x7f09a81033a0 2024-03-18T15:41:18.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a819f960 con 0x7f09a81033a0 2024-03-18T15:41:18.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f099e7fc640 1 -- 172.21.15.96:0/3222590304 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0039040 con 0x7f09a81033a0 2024-03-18T15:41:18.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.166+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a819fea0 con 0x7f09a81033a0 2024-03-18T15:41:18.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.169+0000 7f099e7fc640 1 -- 172.21.15.96:0/3222590304 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f09a0044020 con 0x7f09a81033a0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.169+0000 7f09b1fc0740 1 Processor -- start 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.169+0000 7f09b1fc0740 1 -- start start 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4edfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4ee4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee53b0 0x5632b4ee7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632b4e44490 con 0x5632b4ee0240 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5632b4decc90 con 0x5632b4edf850 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632b4e40530 con 0x5632b4ee53b0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ad170640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4ee4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4edfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ad170640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4ee4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52964/0 (socket says 172.21.15.96:52964) 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ad170640 1 -- 172.21.15.96:0/889042382 learned_addr learned my addr 172.21.15.96:0/889042382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee53b0 0x5632b4ee7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f099e7fc640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 0x7f09940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ad971640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 0x7f09940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 -- 172.21.15.96:0/889042382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee53b0 msgr2=0x5632b4ee7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee53b0 0x5632b4ee7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 -- 172.21.15.96:0/889042382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 msgr2=0x5632b4ee4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4ee4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 -- 172.21.15.96:0/889042382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b4ea7500 con 0x5632b4edf850 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f099e7fc640 1 -- 172.21.15.96:0/3222590304 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f09a00f0900 con 0x7f09a81033a0 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ac96f640 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4edfc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09a8198fc0 tx=0x7f0998004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26d0ddf01a27762 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09ad971640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 0x7f09940a4dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f09a00bd240 tx=0x7f09a00dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.170+0000 7f09937fe640 1 -- 172.21.15.96:0/889042382 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998004050 con 0x5632b4edf850 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09937fe640 1 -- 172.21.15.96:0/889042382 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099800fca0 con 0x5632b4edf850 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09937fe640 1 -- 172.21.15.96:0/889042382 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998003950 con 0x5632b4edf850 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 -- 172.21.15.96:0/889042382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 msgr2=0x5632b4edfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4edfc70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f09a8198fc0 tx=0x7f0998004730 comp rx=0 tx=0).stop 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 -- 172.21.15.96:0/889042382 shutdown_connections 2024-03-18T15:41:18.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee53b0 0x5632b4ee7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4ee4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/889042382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4edfc70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 -- 172.21.15.96:0/889042382 >> 172.21.15.96:0/889042382 conn(0x5632b4edadc0 msgr2=0x5632b4edb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 -- 172.21.15.96:0/889042382 shutdown_connections 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.171+0000 7f09b1fc0740 1 -- 172.21.15.96:0/889042382 wait complete. 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 -- start start 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 Processor -- start 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 0x5632b4ee9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4ee53b0 0x5632b4ef1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632b4ee8360 con 0x5632b4ee0240 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5632b4e00230 con 0x5632b4ee53b0 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632b4ee84e0 con 0x5632b4edf850 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09b1fc0740 10 client.0 _ll_register_callbacks cb 0x5632b4e3c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ad170640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 0x5632b4ee9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ad170640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52976/0 (socket says 172.21.15.96:52976) 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ad170640 1 -- 172.21.15.96:0/1590307185 learned_addr learned my addr 172.21.15.96:0/1590307185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4ee53b0 0x5632b4ef1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 -- 172.21.15.96:0/1590307185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4ee53b0 msgr2=0x5632b4ef1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4ee53b0 0x5632b4ef1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 -- 172.21.15.96:0/1590307185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 msgr2=0x5632b4eea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.172+0000 7f09ac96f640 1 -- 172.21.15.96:0/1590307185 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b4e0f100 con 0x5632b4edf850 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f09ad170640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f09ac96f640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 0x5632b4ee9d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09a8198fc0 tx=0x7f0998009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1127b137c5293570 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800f980 con 0x5632b4edf850 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099800fca0 con 0x5632b4edf850 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0998003b00 con 0x5632b4edf850 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0998003d00 con 0x5632b4edf850 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f09b1fc0740 10 client.25648 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5632b4b583f0 con 0x5632b4edf850 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099800fca0 con 0x5632b4edf850 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.173+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f0998037070 con 0x5632b4edf850 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 10 client.25648 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 10 client.25648 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 20 client.25648 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ef5930 0x5632b4ef7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 4 client.25648 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632b4e519d0 con 0x5632b4ef5930 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09ad170640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ef5930 0x5632b4ef7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f09ad170640 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ef5930 0x5632b4ef7d20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=668edda0c0b6e356 server_cookie=5921f5cb2499915a in_seq=0 out_seq=0 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.174+0000 7f0990ff9640 10 client.25648 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f0990ff9640 1 -- 172.21.15.96:0/1590307185 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632b4e519d0 con 0x5632b4ef5930 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f0990ff9640 10 client.25648 handle_command_reply: tid=0 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 client.25648 shutdown 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 msgr2=0x5632b4ee9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 0x5632b4ee9d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09a8198fc0 tx=0x7f0998009620 comp rx=0 tx=0).stop 2024-03-18T15:41:18.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 shutdown_connections 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ef5930 0x5632b4ef7d20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4ee53b0 0x5632b4ef1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f0990ff9640 0 client.25648 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4ee0240 0x5632b4eea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/1590307185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4edf850 0x5632b4ee9d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.175+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 >> 172.21.15.96:0/1590307185 conn(0x5632b4edadc0 msgr2=0x5632b4ed9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.176+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 shutdown_connections 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.176+0000 7f09b1fc0740 1 -- 172.21.15.96:0/1590307185 wait complete. 2024-03-18T15:41:18.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.176+0000 7f09b1fc0740 20 client.25648 trim_cache size 0 max 16384 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 Processor -- start 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 -- start start 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4ef12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4eea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81a8d90 con 0x5632b4edadc0 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aa990 con 0x5632b4edf850 2024-03-18T15:41:18.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81a9460 con 0x5632b4ee0240 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52982/0 (socket says 172.21.15.96:52982) 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 -- 172.21.15.96:0/762245234 learned_addr learned my addr 172.21.15.96:0/762245234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ad170640 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4ef12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4eea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 -- 172.21.15.96:0/762245234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 msgr2=0x5632b4eea0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4eea0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 -- 172.21.15.96:0/762245234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 msgr2=0x5632b4ef12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4ef12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.189+0000 7f09ac96f640 1 -- 172.21.15.96:0/762245234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b4c227d0 con 0x5632b4edadc0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09ac96f640 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0998021040 tx=0x7f099800f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d012b165b6346ff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f0992ffd640 1 -- 172.21.15.96:0/762245234 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980035f0 con 0x5632b4edadc0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f0992ffd640 1 -- 172.21.15.96:0/762245234 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998034070 con 0x5632b4edadc0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f0992ffd640 1 -- 172.21.15.96:0/762245234 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980302a0 con 0x5632b4edadc0 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 -- 172.21.15.96:0/762245234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 msgr2=0x5632b4ee0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0998021040 tx=0x7f099800f880 comp rx=0 tx=0).stop 2024-03-18T15:41:18.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 -- 172.21.15.96:0/762245234 shutdown_connections 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4eea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4ef12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/762245234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4ee0aa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 -- 172.21.15.96:0/762245234 >> 172.21.15.96:0/762245234 conn(0x5632b4ee8fc0 msgr2=0x5632b4ee99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 -- 172.21.15.96:0/762245234 shutdown_connections 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.190+0000 7f09b1fc0740 1 -- 172.21.15.96:0/762245234 wait complete. 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 -- start start 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 Processor -- start 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4eee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4f59c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aa200 con 0x5632b4edadc0 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81a9b30 con 0x5632b4edf850 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81a6d70 con 0x5632b4ee0240 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09b1fc0740 10 client.0 _ll_register_callbacks cb 0x7f09a40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52990/0 (socket says 172.21.15.96:52990) 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 -- 172.21.15.96:0/2254265094 learned_addr learned my addr 172.21.15.96:0/2254265094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ad170640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4eee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4f59c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 -- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 msgr2=0x5632b4f59c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4f59c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 -- 172.21.15.96:0/2254265094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 msgr2=0x5632b4eee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4eee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.191+0000 7f09ac96f640 1 -- 172.21.15.96:0/2254265094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b4f5a150 con 0x5632b4edadc0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09ac96f640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0998010010 tx=0x7f0998011240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139de00f75847b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800f8f0 con 0x5632b4edadc0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f099800fa90 con 0x5632b4edadc0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0998034070 con 0x5632b4edadc0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f099800fc90 con 0x5632b4edadc0 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 10 client.15813 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5632b4f5b2d0 con 0x5632b4edadc0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998034070 con 0x5632b4edadc0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f0998037070 con 0x5632b4edadc0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 10 client.15813 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 20 client.15813 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ee53b0 0x5632b4f5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 4 client.15813 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632b4e4bd20 con 0x5632b4ee53b0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.192+0000 7f09ad170640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ee53b0 0x5632b4f5dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09ad170640 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ee53b0 0x5632b4f5dbb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b78c4a67d8bd286 server_cookie=ad3458afda8671bb in_seq=0 out_seq=0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f0990ff9640 10 client.15813 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f0990ff9640 1 -- 172.21.15.96:0/2254265094 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632b4e4bd20 con 0x5632b4ee53b0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f0990ff9640 10 client.15813 handle_command_reply: tid=0 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 client.15813 shutdown 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 msgr2=0x5632b4eee1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0998010010 tx=0x7f0998011240 comp rx=0 tx=0).stop 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 shutdown_connections 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5632b4ee53b0 0x5632b4f5dbb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632b4ee0240 0x5632b4f59c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f0990ff9640 0 client.15813 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5632b4edf850 0x5632b4eee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 --2- 172.21.15.96:0/2254265094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632b4edadc0 0x5632b4eee1d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 >> 172.21.15.96:0/2254265094 conn(0x5632b4ee8fc0 msgr2=0x5632b4ee99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 shutdown_connections 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 1 -- 172.21.15.96:0/2254265094 wait complete. 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.193+0000 7f09b1fc0740 20 client.15813 trim_cache size 0 max 16384 2024-03-18T15:41:18.200 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 msgr2=0x7f09940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 0x7f09940a4dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f09a00bd240 tx=0x7f09a00dd040 comp rx=0 tx=0).stop 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 msgr2=0x7f09a819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f09a0037040 tx=0x7f09a0032f40 comp rx=0 tx=0).stop 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 shutdown_connections 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09940a2910 0x7f09940a4dd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a81033a0 0x7f09a819f060 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09a81029e0 0x7f09a8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 --2- 172.21.15.96:0/3222590304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09a81017f0 0x7f09a8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 >> 172.21.15.96:0/3222590304 conn(0x7f09a80fcd80 msgr2=0x7f09a8105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 shutdown_connections 2024-03-18T15:41:18.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.194+0000 7f09af3d4640 1 -- 172.21.15.96:0/3222590304 wait complete. 2024-03-18T15:41:18.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:41:18.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- 172.21.15.96:0/1361041075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060102300 msgr2=0x7fa0601046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- 172.21.15.96:0/1361041075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060102300 0x7fa0601046f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:18.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- 172.21.15.96:0/1361041075 shutdown_connections 2024-03-18T15:41:18.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- 172.21.15.96:0/1361041075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 0x7fa060109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- 172.21.15.96:0/1361041075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060104c30 0x7fa060107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- 172.21.15.96:0/1361041075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060102300 0x7fa0601046f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- 172.21.15.96:0/1361041075 >> 172.21.15.96:0/1361041075 conn(0x7fa0600fbc70 msgr2=0x7fa0600fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- 172.21.15.96:0/1361041075 shutdown_connections 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- 172.21.15.96:0/1361041075 wait complete. 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 Processor -- start 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- start start 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060102300 0x7fa060197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 0x7fa06019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d480 con 0x7fa060104c30 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d300 con 0x7fa060107560 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.311+0000 7fa06804b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d600 con 0x7fa060102300 2024-03-18T15:41:18.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53002/0 (socket says 172.21.15.96:53002) 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 -- 172.21.15.96:0/3092368223 learned_addr learned my addr 172.21.15.96:0/3092368223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa065de7640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060102300 0x7fa060197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 0x7fa06019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 -- 172.21.15.96:0/3092368223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060102300 msgr2=0x7fa060197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060102300 0x7fa060197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 -- 172.21.15.96:0/3092368223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 msgr2=0x7fa06019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 0x7fa06019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 -- 172.21.15.96:0/3092368223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019f7a0 con 0x7fa060104c30 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa0655e6640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9f0 tx=0x7fa05c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa056ffd640 1 -- 172.21.15.96:0/3092368223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00ccc0 con 0x7fa060104c30 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06019fa90 con 0x7fa060104c30 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa056ffd640 1 -- 172.21.15.96:0/3092368223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05c005500 con 0x7fa060104c30 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa056ffd640 1 -- 172.21.15.96:0/3092368223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c010690 con 0x7fa060104c30 2024-03-18T15:41:18.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.312+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06019ff90 con 0x7fa060104c30 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.314+0000 7fa056ffd640 1 -- 172.21.15.96:0/3092368223 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fa05c0046c0 con 0x7fa060104c30 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.314+0000 7fa06ac36740 1 Processor -- start 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 -- start start 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 0x55d2d1f35d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa056ffd640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 0x7fa04c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f363b0 0x55d2d1f38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1e95490 con 0x55d2d1f30850 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1e3dc90 con 0x55d2d1f31240 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa06ac36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1e91530 con 0x55d2d1f363b0 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0655e6640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 0x7fa04c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53016/0 (socket says 172.21.15.96:53016) 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 -- 172.21.15.96:0/2303342889 learned_addr learned my addr 172.21.15.96:0/2303342889 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa065de7640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 0x55d2d1f35d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f363b0 0x55d2d1f38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa056ffd640 1 -- 172.21.15.96:0/3092368223 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa05c0cef50 con 0x7fa060104c30 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 -- 172.21.15.96:0/2303342889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f363b0 msgr2=0x55d2d1f38840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f363b0 0x55d2d1f38840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 -- 172.21.15.96:0/2303342889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 msgr2=0x55d2d1f35d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 0x55d2d1f35d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa0665e8640 1 -- 172.21.15.96:0/2303342889 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d1ef8500 con 0x55d2d1f30850 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.315+0000 7fa065de7640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 0x55d2d1f35d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa0655e6640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 0x7fa04c0a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa05c09b6f0 tx=0x7fa05c086040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa058005c40 tx=0x7fa058009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30632ea6bdb7c1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa04bfff640 1 -- 172.21.15.96:0/2303342889 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058003970 con 0x55d2d1f30850 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa04bfff640 1 -- 172.21.15.96:0/2303342889 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580118a0 con 0x55d2d1f30850 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa04bfff640 1 -- 172.21.15.96:0/2303342889 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058004590 con 0x55d2d1f30850 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 -- 172.21.15.96:0/2303342889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 msgr2=0x55d2d1f30c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa058005c40 tx=0x7fa058009920 comp rx=0 tx=0).stop 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 -- 172.21.15.96:0/2303342889 shutdown_connections 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f363b0 0x55d2d1f38840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f31240 0x55d2d1f35d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2303342889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1f30c70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.316+0000 7fa06ac36740 1 -- 172.21.15.96:0/2303342889 >> 172.21.15.96:0/2303342889 conn(0x55d2d1f2bdc0 msgr2=0x55d2d1f2c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- 172.21.15.96:0/2303342889 shutdown_connections 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- 172.21.15.96:0/2303342889 wait complete. 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- start start 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 Processor -- start 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f42250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1f39300 con 0x55d2d1f31240 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1e51230 con 0x55d2d1f363b0 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d2d1f39480 con 0x55d2d1f30850 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa065de7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa06ac36740 10 client.0 _ll_register_callbacks cb 0x55d2d1e8d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa065de7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53018/0 (socket says 172.21.15.96:53018) 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa065de7640 1 -- 172.21.15.96:0/3663349276 learned_addr learned my addr 172.21.15.96:0/3663349276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.317+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f42250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 -- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 msgr2=0x55d2d1f3ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 -- 172.21.15.96:0/3663349276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 msgr2=0x55d2d1f42250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f42250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 -- 172.21.15.96:0/3663349276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d1e60100 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa065de7640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa050036040 tx=0x7fa050004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e330e8443301aad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0497fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050005d50 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa050005230 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0580118a0 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa050005430 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa06ac36740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2d1ba93f0 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.318+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0580118a0 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa05004e070 con 0x55d2d1f31240 2024-03-18T15:41:18.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 10 client.15825 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 20 client.15825 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1f48fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 4 client.15825 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d1ea29d0 con 0x55d2d1f46b40 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1f48fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa0497fa640 10 client.15825 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:18.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.319+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1f48fa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4592d3963eb83952 server_cookie=d86c5cbc02ca70d4 in_seq=0 out_seq=0 2024-03-18T15:41:18.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.325+0000 7fa0497fa640 1 -- 172.21.15.96:0/3663349276 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d2d1ea29d0 con 0x55d2d1f46b40 2024-03-18T15:41:18.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.325+0000 7fa0497fa640 10 client.15825 handle_command_reply: tid=0 2024-03-18T15:41:18.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.325+0000 7fa06ac36740 1 client.15825 shutdown 2024-03-18T15:41:18.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 msgr2=0x55d2d1f3b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa050036040 tx=0x7fa050004250 comp rx=0 tx=0).stop 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 shutdown_connections 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1f48fa0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f42250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa0497fa640 0 client.15825 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3663349276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 >> 172.21.15.96:0/3663349276 conn(0x55d2d1f2bdc0 msgr2=0x55d2d1f2afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 shutdown_connections 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 1 -- 172.21.15.96:0/3663349276 wait complete. 2024-03-18T15:41:18.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.326+0000 7fa06ac36740 20 client.15825 trim_cache size 0 max 16384 2024-03-18T15:41:18.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 Processor -- start 2024-03-18T15:41:18.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 -- start start 2024-03-18T15:41:18.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f2a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601a8e70 con 0x55d2d1f31240 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601aaa70 con 0x55d2d1f363b0 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa06ac36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601a9540 con 0x55d2d1f30850 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa065de7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa065de7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53028/0 (socket says 172.21.15.96:53028) 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa065de7640 1 -- 172.21.15.96:0/2109903589 learned_addr learned my addr 172.21.15.96:0/2109903589 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f2a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.339+0000 7fa0665e8640 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 -- 172.21.15.96:0/2109903589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 msgr2=0x55d2d1f3ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 -- 172.21.15.96:0/2109903589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 msgr2=0x55d2d1f2a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f2a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 -- 172.21.15.96:0/2109903589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d1c737d0 con 0x55d2d1f31240 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa065de7640 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa050005cc0 tx=0x7fa050009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117bedd8f8891f50 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa04b7fe640 1 -- 172.21.15.96:0/2109903589 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050054070 con 0x55d2d1f31240 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa04b7fe640 1 -- 172.21.15.96:0/2109903589 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05004b070 con 0x55d2d1f31240 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa04b7fe640 1 -- 172.21.15.96:0/2109903589 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050003c20 con 0x55d2d1f31240 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 -- 172.21.15.96:0/2109903589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 msgr2=0x55d2d1f3b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa050005cc0 tx=0x7fa050009830 comp rx=0 tx=0).stop 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 -- 172.21.15.96:0/2109903589 shutdown_connections 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1f2a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f31240 0x55d2d1f3b220 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 --2- 172.21.15.96:0/2109903589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f30850 0x55d2d1f3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.340+0000 7fa06ac36740 1 -- 172.21.15.96:0/2109903589 >> 172.21.15.96:0/2109903589 conn(0x55d2d1f2bdc0 msgr2=0x55d2d1f2c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- 172.21.15.96:0/2109903589 shutdown_connections 2024-03-18T15:41:18.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- 172.21.15.96:0/2109903589 wait complete. 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- start start 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 Processor -- start 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f31240 0x55d2d1fa8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1faf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601aa2e0 con 0x55d2d1f30850 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601a9c10 con 0x55d2d1f363b0 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601a6e50 con 0x55d2d1f31240 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa06ac36740 10 client.0 _ll_register_callbacks cb 0x7fa050009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa0665e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa0665e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53030/0 (socket says 172.21.15.96:53030) 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa0665e8640 1 -- 172.21.15.96:0/3935740822 learned_addr learned my addr 172.21.15.96:0/3935740822 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa065de7640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f31240 0x55d2d1fa8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.341+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1faf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 -- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f31240 msgr2=0x55d2d1fa8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f31240 0x55d2d1fa8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 -- 172.21.15.96:0/3935740822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 msgr2=0x55d2d1faf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1faf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 -- 172.21.15.96:0/3935740822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d1fafcb0 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0665e8640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa058009ae0 tx=0x7fa058022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed29962bd5e1b220 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058013030 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa0580131d0 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05004b070 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa0580313d0 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2d1fb0fb0 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa05004b070 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa058005270 con 0x55d2d1f30850 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 10 client.15831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 20 client.15831 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1fb3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 4 client.15831 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.342+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2d1e9cd20 con 0x55d2d1f46b40 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.343+0000 7fa065de7640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1fb3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.343+0000 7fa0497fa640 10 client.15831 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:18.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.343+0000 7fa065de7640 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1fb3860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b53d2246ac52263 server_cookie=13ab20574104759f in_seq=0 out_seq=0 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.350+0000 7fa0497fa640 1 -- 172.21.15.96:0/3935740822 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2d1e9cd20 con 0x55d2d1f46b40 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.350+0000 7fa0497fa640 10 client.15831 handle_command_reply: tid=0 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.350+0000 7fa06ac36740 1 client.15831 shutdown 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 msgr2=0x55d2d1fa8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa058009ae0 tx=0x7fa058022040 comp rx=0 tx=0).stop 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 shutdown_connections 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55d2d1f46b40 0x55d2d1fb3860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d2d1f363b0 0x55d2d1faf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa0497fa640 0 client.15831 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2d1f31240 0x55d2d1fa8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 --2- 172.21.15.96:0/3935740822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2d1f30850 0x55d2d1fa8140 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 >> 172.21.15.96:0/3935740822 conn(0x55d2d1f2bdc0 msgr2=0x55d2d1f3ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 shutdown_connections 2024-03-18T15:41:18.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 1 -- 172.21.15.96:0/3935740822 wait complete. 2024-03-18T15:41:18.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06ac36740 20 client.15831 trim_cache size 0 max 16384 2024-03-18T15:41:18.353 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 msgr2=0x7fa04c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 0x7fa04c0a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa05c09b6f0 tx=0x7fa05c086040 comp rx=0 tx=0).stop 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 msgr2=0x7fa0601980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00e9f0 tx=0x7fa05c00edf0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 shutdown_connections 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa04c0a2890 0x7fa04c0a4d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa060107560 0x7fa06019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa060104c30 0x7fa0601980c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 --2- 172.21.15.96:0/3092368223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060102300 0x7fa060197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.351+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 >> 172.21.15.96:0/3092368223 conn(0x7fa0600fbc70 msgr2=0x7fa060102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.352+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 shutdown_connections 2024-03-18T15:41:18.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.352+0000 7fa06804b640 1 -- 172.21.15.96:0/3092368223 wait complete. 2024-03-18T15:41:18.364 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:41:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- 172.21.15.96:0/2473956865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 msgr2=0x7fef74102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- 172.21.15.96:0/2473956865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef74102480 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fef70009900 tx=0x7fef7002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- 172.21.15.96:0/2473956865 shutdown_connections 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- 172.21.15.96:0/2473956865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef74105370 0x7fef741057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- 172.21.15.96:0/2473956865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef741029c0 0x7fef74104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- 172.21.15.96:0/2473956865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef74102480 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- 172.21.15.96:0/2473956865 >> 172.21.15.96:0/2473956865 conn(0x7fef740fbc70 msgr2=0x7fef740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- 172.21.15.96:0/2473956865 shutdown_connections 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- 172.21.15.96:0/2473956865 wait complete. 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 Processor -- start 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- start start 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef741029c0 0x7fef7419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef74105370 0x7fef741a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef74113a50 con 0x7fef74100050 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef741138d0 con 0x7fef741029c0 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.472+0000 7fef7b72f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef74113bd0 con 0x7fef74105370 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53052/0 (socket says 172.21.15.96:53052) 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 -- 172.21.15.96:0/1114299423 learned_addr learned my addr 172.21.15.96:0/1114299423 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef74105370 0x7fef741a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef78cca640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef741029c0 0x7fef7419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 -- 172.21.15.96:0/1114299423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef74105370 msgr2=0x7fef741a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef74105370 0x7fef741a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 -- 172.21.15.96:0/1114299423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef741029c0 msgr2=0x7fef7419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef741029c0 0x7fef7419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 -- 172.21.15.96:0/1114299423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741a5d40 con 0x7fef74100050 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef794cb640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef70009230 tx=0x7fef7002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef6a7fc640 1 -- 172.21.15.96:0/1114299423 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70041070 con 0x7fef74100050 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef6a7fc640 1 -- 172.21.15.96:0/1114299423 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef70004450 con 0x7fef74100050 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef741a5fd0 con 0x7fef74100050 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef6a7fc640 1 -- 172.21.15.96:0/1114299423 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70003e20 con 0x7fef74100050 2024-03-18T15:41:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.473+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef741a63d0 con 0x7fef74100050 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.475+0000 7fef6a7fc640 1 -- 172.21.15.96:0/1114299423 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fef70044070 con 0x7fef74100050 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 Processor -- start 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 -- start start 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072b850 0x563fc072bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc0730d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fc0690490 con 0x563fc072b850 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563fc0638c90 con 0x563fc072c240 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563fc068c530 con 0x563fc07313b0 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef6a7fc640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 0x7fef600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef794cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc0730d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:47954/0 (socket says 172.21.15.96:47954) 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 -- 172.21.15.96:0/3590080236 learned_addr learned my addr 172.21.15.96:0/3590080236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef79ccc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072b850 0x563fc072bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 0x7fef600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef6a7fc640 1 -- 172.21.15.96:0/1114299423 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fef700f0680 con 0x7fef74100050 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 -- 172.21.15.96:0/3590080236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 msgr2=0x563fc0730d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc0730d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 -- 172.21.15.96:0/3590080236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072b850 msgr2=0x563fc072bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072b850 0x563fc072bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.476+0000 7fef78cca640 1 -- 172.21.15.96:0/3590080236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc06f3500 con 0x563fc07313b0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef78cca640 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef64009ca0 tx=0x7fef64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ee1128279a2170 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 0x7fef600a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x563fc072caa0 tx=0x7fef6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef5f7fe640 1 -- 172.21.15.96:0/3590080236 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6401e070 con 0x563fc07313b0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef5f7fe640 1 -- 172.21.15.96:0/3590080236 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef64009e50 con 0x563fc07313b0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef5f7fe640 1 -- 172.21.15.96:0/3590080236 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6400e970 con 0x563fc07313b0 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef7e3f0740 1 -- 172.21.15.96:0/3590080236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 msgr2=0x563fc0733840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.477+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef64009ca0 tx=0x7fef64011810 comp rx=0 tx=0).stop 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- 172.21.15.96:0/3590080236 shutdown_connections 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc0733840 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc0730d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/3590080236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072b850 0x563fc072bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- 172.21.15.96:0/3590080236 >> 172.21.15.96:0/3590080236 conn(0x563fc0726dc0 msgr2=0x563fc07271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- 172.21.15.96:0/3590080236 shutdown_connections 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- 172.21.15.96:0/3590080236 wait complete. 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- start start 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 Processor -- start 2024-03-18T15:41:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0735d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc073d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fc0734360 con 0x563fc072c240 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563fc064c230 con 0x563fc072b850 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563fc07344e0 con 0x563fc07313b0 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.478+0000 7fef7e3f0740 10 client.0 _ll_register_callbacks cb 0x563fc0688010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef794cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0735d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53058/0 (socket says 172.21.15.96:53058) 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 -- 172.21.15.96:0/4117993775 learned_addr learned my addr 172.21.15.96:0/4117993775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef79ccc640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc073d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 -- 172.21.15.96:0/4117993775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 msgr2=0x563fc073d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc073d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 -- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 msgr2=0x563fc0735d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0735d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 -- 172.21.15.96:0/4117993775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc065b100 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef794cb640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0735d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef78cca640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef64005720 tx=0x7fef64005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18944c76fdff5a43 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64027030 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fef64004020 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef64009e50 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef6401e070 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef7e3f0740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fc03a43f0 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.479+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef64009e50 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fef64004a50 con 0x563fc072c240 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 10 client.15843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:41:18.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 20 client.15843 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc0741810 0x563fc0743c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 4 client.15843 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc069d9d0 con 0x563fc0741810 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef794cb640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc0741810 0x563fc0743c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef5cff9640 10 client.15843 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.480+0000 7fef794cb640 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc0741810 0x563fc0743c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e5e2d8a9888c2fd server_cookie=9b07f3afe201eb71 in_seq=0 out_seq=0 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef5cff9640 1 -- 172.21.15.96:0/4117993775 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563fc069d9d0 con 0x563fc0741810 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef5cff9640 10 client.15843 handle_command_reply: tid=0 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 client.15843 shutdown 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 msgr2=0x563fc07362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fef64005720 tx=0x7fef64005750 comp rx=0 tx=0).stop 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 shutdown_connections 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc073d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc07362a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc0741810 0x563fc0743c80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4117993775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0735d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef5cff9640 0 client.15843 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 >> 172.21.15.96:0/4117993775 conn(0x563fc0726dc0 msgr2=0x563fc0725fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 shutdown_connections 2024-03-18T15:41:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4117993775 wait complete. 2024-03-18T15:41:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.481+0000 7fef7e3f0740 20 client.15843 trim_cache size 0 max 16384 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 Processor -- start 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 -- start start 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0734880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc072a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef741af340 con 0x563fc072c240 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef741b0f40 con 0x563fc072b850 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef741afa10 con 0x563fc07313b0 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53062/0 (socket says 172.21.15.96:53062) 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 -- 172.21.15.96:0/4057026544 learned_addr learned my addr 172.21.15.96:0/4057026544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef794cb640 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0734880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef79ccc640 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc072a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 -- 172.21.15.96:0/4057026544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 msgr2=0x563fc072a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc072a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 -- 172.21.15.96:0/4057026544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 msgr2=0x563fc0734880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0734880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.495+0000 7fef78cca640 1 -- 172.21.15.96:0/4057026544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc046e7d0 con 0x563fc072c240 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef78cca640 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef640059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2842059b6f9d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef5effd640 1 -- 172.21.15.96:0/4057026544 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64027040 con 0x563fc072c240 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef5effd640 1 -- 172.21.15.96:0/4057026544 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6401f040 con 0x563fc072c240 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef5effd640 1 -- 172.21.15.96:0/4057026544 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6400eda0 con 0x563fc072c240 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4057026544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 msgr2=0x563fc073d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fef64006010 tx=0x7fef640059b0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4057026544 shutdown_connections 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc07313b0 0x563fc072a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc072c240 0x563fc073d260 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/4057026544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072b850 0x563fc0734880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4057026544 >> 172.21.15.96:0/4057026544 conn(0x563fc0726dc0 msgr2=0x563fc072a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4057026544 shutdown_connections 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.496+0000 7fef7e3f0740 1 -- 172.21.15.96:0/4057026544 wait complete. 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 -- start start 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 Processor -- start 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc072b850 0x563fc073aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc073b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef741b07b0 con 0x563fc07313b0 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef741b00e0 con 0x563fc072c240 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef741ad320 con 0x563fc072b850 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef7e3f0740 10 client.0 _ll_register_callbacks cb 0x7fef6401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53076/0 (socket says 172.21.15.96:53076) 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 -- 172.21.15.96:0/1334153439 learned_addr learned my addr 172.21.15.96:0/1334153439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef794cb640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc072b850 0x563fc073aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef78cca640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc073b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 -- 172.21.15.96:0/1334153439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc072b850 msgr2=0x563fc073aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc072b850 0x563fc073aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 -- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 msgr2=0x563fc073b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc073b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.497+0000 7fef79ccc640 1 -- 172.21.15.96:0/1334153439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc073c690 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef79ccc640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef6c003ec0 tx=0x7fef6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3068d62ba419e82 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c004b20 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef6401f040 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fef6c004cc0 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef6c005020 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fc07a7e00 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6401f040 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fef6c028070 con 0x563fc07313b0 2024-03-18T15:41:18.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 10 client.15849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 20 client.15849 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc07a88b0 0x563fc07aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 4 client.15849 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.498+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc0697d20 con 0x563fc07a88b0 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef794cb640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc07a88b0 0x563fc07aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef5cff9640 10 client.15849 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef794cb640 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc07a88b0 0x563fc07aacd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4684ceaa5828175 server_cookie=3bef1aa52dc1251e in_seq=0 out_seq=0 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef5cff9640 1 -- 172.21.15.96:0/1334153439 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc0697d20 con 0x563fc07a88b0 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef5cff9640 10 client.15849 handle_command_reply: tid=0 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 client.15849 shutdown 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 msgr2=0x563fc073c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fef6c003ec0 tx=0x7fef6c004a40 comp rx=0 tx=0).stop 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 shutdown_connections 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fc07313b0 0x563fc073c0a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563fc07a88b0 0x563fc07aacd0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563fc072c240 0x563fc073b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 --2- 172.21.15.96:0/1334153439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563fc072b850 0x563fc073aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef5cff9640 0 client.15849 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 >> 172.21.15.96:0/1334153439 conn(0x563fc0726dc0 msgr2=0x563fc072a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 shutdown_connections 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 1 -- 172.21.15.96:0/1334153439 wait complete. 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.499+0000 7fef7e3f0740 20 client.15849 trim_cache size 0 max 16384 2024-03-18T15:41:18.509 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 msgr2=0x7fef600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 0x7fef600a4d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x563fc072caa0 tx=0x7fef6c009500 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 msgr2=0x7fef7419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fef70009230 tx=0x7fef7002c650 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 shutdown_connections 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fef600a2890 0x7fef600a4d50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef74105370 0x7fef741a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef741029c0 0x7fef7419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 --2- 172.21.15.96:0/1114299423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef74100050 0x7fef7419e170 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 >> 172.21.15.96:0/1114299423 conn(0x7fef740fbc70 msgr2=0x7fef74103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 shutdown_connections 2024-03-18T15:41:18.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.500+0000 7fef7b72f640 1 -- 172.21.15.96:0/1114299423 wait complete. 2024-03-18T15:41:18.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 -- 172.21.15.96:0/2262782928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 msgr2=0x7fefc8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 --2- 172.21.15.96:0/2262782928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8101d50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009900 tx=0x7fefb802bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 -- 172.21.15.96:0/2262782928 shutdown_connections 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 --2- 172.21.15.96:0/2262782928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 0x7fefc8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 --2- 172.21.15.96:0/2262782928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8101d50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 --2- 172.21.15.96:0/2262782928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 0x7fefc8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 -- 172.21.15.96:0/2262782928 >> 172.21.15.96:0/2262782928 conn(0x7fefc80fbc70 msgr2=0x7fefc80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 -- 172.21.15.96:0/2262782928 shutdown_connections 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 -- 172.21.15.96:0/2262782928 wait complete. 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.619+0000 7fefd0497640 1 Processor -- start 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 -- start start 2024-03-18T15:41:18.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 0x7fefc8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 0x7fefc819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810d470 con 0x7fefc81018d0 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810d2f0 con 0x7fefc81006e0 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810d5f0 con 0x7fefc8102290 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53096/0 (socket says 172.21.15.96:53096) 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 -- 172.21.15.96:0/4196193852 learned_addr learned my addr 172.21.15.96:0/4196193852 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefce233640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 0x7fefc8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcea34640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 0x7fefc819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 -- 172.21.15.96:0/4196193852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 msgr2=0x7fefc819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 0x7fefc819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 -- 172.21.15.96:0/4196193852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 msgr2=0x7fefc8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 0x7fefc8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 -- 172.21.15.96:0/4196193852 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc819f720 con 0x7fefc81018d0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefcda32640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009230 tx=0x7fefb802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefbf7fe640 1 -- 172.21.15.96:0/4196193852 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8041070 con 0x7fefc81018d0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefbf7fe640 1 -- 172.21.15.96:0/4196193852 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefb8004450 con 0x7fefc81018d0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.620+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc819f9b0 con 0x7fefc81018d0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.621+0000 7fefbf7fe640 1 -- 172.21.15.96:0/4196193852 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8003e20 con 0x7fefc81018d0 2024-03-18T15:41:18.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.621+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc819fea0 con 0x7fefc81018d0 2024-03-18T15:41:18.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.623+0000 7fefbf7fe640 1 -- 172.21.15.96:0/4196193852 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fefb8044070 con 0x7fefc81018d0 2024-03-18T15:41:18.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.623+0000 7fefd3082740 1 Processor -- start 2024-03-18T15:41:18.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.623+0000 7fefd3082740 1 -- start start 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.623+0000 7fefd3082740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 0x56454fa83080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.623+0000 7fefd3082740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa88150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefbf7fe640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 0x7fefb40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefd3082740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa8ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefd3082740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56454f9e3490 con 0x56454fa82c60 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefd3082740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56454f98bc90 con 0x56454fa83650 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefd3082740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56454f9df530 con 0x56454fa887c0 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcda32640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 0x7fefb40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 0x56454fa83080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 0x56454fa83080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53102/0 (socket says 172.21.15.96:53102) 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcea34640 1 -- 172.21.15.96:0/2554546282 learned_addr learned my addr 172.21.15.96:0/2554546282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa88150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcea34640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa8ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefbf7fe640 1 -- 172.21.15.96:0/4196193852 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fefb80f06a0 con 0x7fefc81018d0 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 -- 172.21.15.96:0/2554546282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 msgr2=0x56454fa8ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa8ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 -- 172.21.15.96:0/2554546282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 msgr2=0x56454fa83080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 0x56454fa83080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 -- 172.21.15.96:0/2554546282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454fa46500 con 0x56454fa83650 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcea34640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefcda32640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 0x7fefb40a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefb80b9a40 tx=0x7fefb80b8bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefce233640 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa88150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009bf0 tx=0x7fefc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584d01746e026a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefbcbfe640 1 -- 172.21.15.96:0/2554546282 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4004050 con 0x56454fa83650 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefbcbfe640 1 -- 172.21.15.96:0/2554546282 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc4005500 con 0x56454fa83650 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.624+0000 7fefbcbfe640 1 -- 172.21.15.96:0/2554546282 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4004d80 con 0x56454fa83650 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- 172.21.15.96:0/2554546282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 msgr2=0x56454fa88150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa88150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009bf0 tx=0x7fefc4011810 comp rx=0 tx=0).stop 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- 172.21.15.96:0/2554546282 shutdown_connections 2024-03-18T15:41:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa8ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa88150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 --2- 172.21.15.96:0/2554546282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa82c60 0x56454fa83080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- 172.21.15.96:0/2554546282 >> 172.21.15.96:0/2554546282 conn(0x56454fa79dc0 msgr2=0x56454fa7a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- 172.21.15.96:0/2554546282 shutdown_connections 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- 172.21.15.96:0/2554546282 wait complete. 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.625+0000 7fefd3082740 1 -- start start 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 Processor -- start 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56454fa8b770 con 0x56454fa887c0 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56454f99f230 con 0x56454fa82c60 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56454fa8b8f0 con 0x56454fa83650 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefd3082740 10 client.0 _ll_register_callbacks cb 0x56454f9db010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53106/0 (socket says 172.21.15.96:53106) 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 -- 172.21.15.96:0/442206068 learned_addr learned my addr 172.21.15.96:0/442206068 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefce233640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 -- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa83650 msgr2=0x56454fa8d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 -- 172.21.15.96:0/442206068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 msgr2=0x56454fa8d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 -- 172.21.15.96:0/442206068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454f9ae100 con 0x56454fa887c0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefcea34640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefc0013e90 tx=0x7fefc0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28baeef16e4f5363 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.626+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0004b20 con 0x56454fa887c0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc4005500 con 0x56454fa887c0 2024-03-18T15:41:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fefc0004cc0 con 0x56454fa887c0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefc0029050 con 0x56454fa887c0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 10 client.15861 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56454f6f73f0 con 0x56454fa887c0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc4005500 con 0x56454fa887c0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fefc00046d0 con 0x56454fa887c0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 20 client.15861 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454fa98f50 0x56454fa9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 4 client.15861 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56454f9f09d0 con 0x56454fa98f50 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.627+0000 7fefce233640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454fa98f50 0x56454fa9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.628+0000 7fefce233640 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454fa98f50 0x56454fa9b3f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b66dbb72e2ad7b7 server_cookie=8675f365b9a7a14e in_seq=0 out_seq=0 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.628+0000 7fefb1ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.628+0000 7fefb1ffb640 1 -- 172.21.15.96:0/442206068 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56454f9f09d0 con 0x56454fa98f50 2024-03-18T15:41:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.628+0000 7fefb1ffb640 10 client.15861 handle_command_reply: tid=0 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.628+0000 7fefd3082740 1 client.15861 shutdown 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 msgr2=0x56454fa946d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fefc0013e90 tx=0x7fefc0015040 comp rx=0 tx=0).stop 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 shutdown_connections 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454fa98f50 0x56454fa9b3f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa946d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefb1ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 --2- 172.21.15.96:0/442206068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 >> 172.21.15.96:0/442206068 conn(0x56454fa79dc0 msgr2=0x56454fa78fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 shutdown_connections 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 1 -- 172.21.15.96:0/442206068 wait complete. 2024-03-18T15:41:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.629+0000 7fefd3082740 20 client.15861 trim_cache size 0 max 16384 2024-03-18T15:41:18.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 Processor -- start 2024-03-18T15:41:18.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 -- start start 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa94690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a6ba0 con 0x56454fa83650 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a87a0 con 0x56454fa82c60 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefd3082740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a7270 con 0x56454fa887c0 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefce233640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefce233640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53112/0 (socket says 172.21.15.96:53112) 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefce233640 1 -- 172.21.15.96:0/3050184220 learned_addr learned my addr 172.21.15.96:0/3050184220 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa94690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 -- 172.21.15.96:0/3050184220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 msgr2=0x56454fa94690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa94690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 -- 172.21.15.96:0/3050184220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 msgr2=0x56454fa8d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefcea34640 1 -- 172.21.15.96:0/3050184220 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454f7c17d0 con 0x56454fa82c60 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.642+0000 7fefce233640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefcea34640 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009690 tx=0x7fefc0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da5896b472d223b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefb3fff640 1 -- 172.21.15.96:0/3050184220 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0009e00 con 0x56454fa82c60 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefb3fff640 1 -- 172.21.15.96:0/3050184220 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc000f8e0 con 0x56454fa82c60 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefb3fff640 1 -- 172.21.15.96:0/3050184220 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc0004000 con 0x56454fa82c60 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 -- 172.21.15.96:0/3050184220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 msgr2=0x56454fa8d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009690 tx=0x7fefc0009500 comp rx=0 tx=0).stop 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 -- 172.21.15.96:0/3050184220 shutdown_connections 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa887c0 0x56454fa94690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa83650 0x56454fa8d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 --2- 172.21.15.96:0/3050184220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa82c60 0x56454fa8d150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 -- 172.21.15.96:0/3050184220 >> 172.21.15.96:0/3050184220 conn(0x56454fa79dc0 msgr2=0x56454fa7a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 -- 172.21.15.96:0/3050184220 shutdown_connections 2024-03-18T15:41:18.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.643+0000 7fefd3082740 1 -- 172.21.15.96:0/3050184220 wait complete. 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 -- start start 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 Processor -- start 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa82c60 0x56454fa91ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a8010 con 0x56454fa887c0 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a7940 con 0x56454fa83650 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc81a4b80 con 0x56454fa82c60 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefd3082740 10 client.0 _ll_register_callbacks cb 0x7fefc0013140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefcea34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53124/0 (socket says 172.21.15.96:53124) 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefcea34640 1 -- 172.21.15.96:0/2234435587 learned_addr learned my addr 172.21.15.96:0/2234435587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefcea34640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa82c60 0x56454fa91ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 -- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa82c60 msgr2=0x56454fa91ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa82c60 0x56454fa91ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 -- 172.21.15.96:0/2234435587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 msgr2=0x56454fa93010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefce233640 1 -- 172.21.15.96:0/2234435587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56454fa93600 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.644+0000 7fefcea34640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefce233640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa92000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefc4011d90 tx=0x7fefc4005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb65da0bca08ff7a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc402ecb0 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc000f8e0 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fefc402e2a0 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fefc403c030 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 10 client.25658 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56454fa94900 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc000f8e0 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fefc4019ad0 con 0x56454fa83650 2024-03-18T15:41:18.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 10 client.25658 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 20 client.25658 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454faff7c0 0x56454fb01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 4 client.25658 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.645+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56454f9ead20 con 0x56454faff7c0 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefcea34640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454faff7c0 0x56454fb01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefb1ffb640 10 client.25658 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefcea34640 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454faff7c0 0x56454fb01be0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb583b70edf0696 server_cookie=ff2e48bd729dd72a in_seq=0 out_seq=0 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefb1ffb640 1 -- 172.21.15.96:0/2234435587 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56454f9ead20 con 0x56454faff7c0 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefb1ffb640 10 client.25658 handle_command_reply: tid=0 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 client.25658 shutdown 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 msgr2=0x56454fa92000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa92000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefc4011d90 tx=0x7fefc4005d70 comp rx=0 tx=0).stop 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 shutdown_connections 2024-03-18T15:41:18.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56454faff7c0 0x56454fb01be0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56454fa887c0 0x56454fa93010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefb1ffb640 0 client.25658 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56454fa83650 0x56454fa92000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 --2- 172.21.15.96:0/2234435587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56454fa82c60 0x56454fa91ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 >> 172.21.15.96:0/2234435587 conn(0x56454fa79dc0 msgr2=0x56454fa84bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.646+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 shutdown_connections 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd3082740 1 -- 172.21.15.96:0/2234435587 wait complete. 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd3082740 20 client.25658 trim_cache size 0 max 16384 2024-03-18T15:41:18.654 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 msgr2=0x7fefb40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 0x7fefb40a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefb80b9a40 tx=0x7fefb80b8bf0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 msgr2=0x7fefc8198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009230 tx=0x7fefb802c650 comp rx=0 tx=0).stop 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 shutdown_connections 2024-03-18T15:41:18.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fefb40a2890 0x7fefb40a4d50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc8102290 0x7fefc819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefc81018d0 0x7fefc8198040 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 --2- 172.21.15.96:0/4196193852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fefc81006e0 0x7fefc8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 >> 172.21.15.96:0/4196193852 conn(0x7fefc80fbc70 msgr2=0x7fefc8104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 shutdown_connections 2024-03-18T15:41:18.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:18.647+0000 7fefd0497640 1 -- 172.21.15.96:0/4196193852 wait complete. 2024-03-18T15:41:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:18 smithi193 ceph-mon[36702]: pgmap v388: 97 pgs: 97 active+clean; 17 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-03-18T15:41:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:18 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3662929671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:18 smithi096 ceph-mon[31930]: pgmap v388: 97 pgs: 97 active+clean; 17 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-03-18T15:41:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:18 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3662929671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:18 smithi137 ceph-mon[35868]: pgmap v388: 97 pgs: 97 active+clean; 17 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-03-18T15:41:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:18 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3662929671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:19.659 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:41:19.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:41:19.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 -- 172.21.15.96:0/273116301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230102a70 msgr2=0x7f0230102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 --2- 172.21.15.96:0/273116301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230102a70 0x7f0230102ef0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0218009860 tx=0x7f021802bb70 comp rx=0 tx=0).stop 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 -- 172.21.15.96:0/273116301 shutdown_connections 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 --2- 172.21.15.96:0/273116301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0230103430 0x7f0230109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 --2- 172.21.15.96:0/273116301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230102a70 0x7f0230102ef0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 --2- 172.21.15.96:0/273116301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02300ffaa0 0x7f02300ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 -- 172.21.15.96:0/273116301 >> 172.21.15.96:0/273116301 conn(0x7f02300fb3d0 msgr2=0x7f02300fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 -- 172.21.15.96:0/273116301 shutdown_connections 2024-03-18T15:41:19.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.769+0000 7f023484b640 1 -- 172.21.15.96:0/273116301 wait complete. 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 Processor -- start 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 -- start start 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 0x7f0230197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 0x7f0230198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 0x7f023019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f023010d460 con 0x7f0230102a70 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f023010d2e0 con 0x7f0230103430 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f023484b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f023010d5e0 con 0x7f02300ffaa0 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f0227fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 0x7f0230198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f022e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 0x7f0230197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f022ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 0x7f023019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f0227fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 0x7f0230198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53136/0 (socket says 172.21.15.96:53136) 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.770+0000 7f0227fff640 1 -- 172.21.15.96:0/536927584 learned_addr learned my addr 172.21.15.96:0/536927584 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 -- 172.21.15.96:0/536927584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 msgr2=0x7f023019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 0x7f023019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 -- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 msgr2=0x7f0230198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 0x7f0230198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 -- 172.21.15.96:0/536927584 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023019f720 con 0x7f02300ffaa0 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022ed9d640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 0x7f023019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f022e59c640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 0x7f0230197b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0230068740 tx=0x7f022000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f02277fe640 1 -- 172.21.15.96:0/536927584 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220004440 con 0x7f02300ffaa0 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02301008c0 con 0x7f02300ffaa0 2024-03-18T15:41:19.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f02277fe640 1 -- 172.21.15.96:0/536927584 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f022000cda0 con 0x7f02300ffaa0 2024-03-18T15:41:19.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0230100e60 con 0x7f02300ffaa0 2024-03-18T15:41:19.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.771+0000 7f02277fe640 1 -- 172.21.15.96:0/536927584 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200196e0 con 0x7f02300ffaa0 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.774+0000 7f02277fe640 1 -- 172.21.15.96:0/536927584 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f0220019880 con 0x7f02300ffaa0 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.774+0000 7f02277fe640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 0x7f021c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.774+0000 7f02277fe640 1 -- 172.21.15.96:0/536927584 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f02200cf030 con 0x7f02300ffaa0 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.774+0000 7f0227fff640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 0x7f021c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 Processor -- start 2024-03-18T15:41:19.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 -- start start 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f0227fff640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 0x7f021c0a4d50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0230199010 tx=0x7f0218034ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b75380 0x556226b79e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b7c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556226ad90d0 con 0x556226b74990 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556226a81c70 con 0x556226b7a4f0 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.775+0000 7f023743c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556226ad5170 con 0x556226b75380 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53148/0 (socket says 172.21.15.96:53148) 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 -- 172.21.15.96:0/802177857 learned_addr learned my addr 172.21.15.96:0/802177857 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f0227fff640 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b75380 0x556226b79e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022ed9d640 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b7c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 -- 172.21.15.96:0/802177857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b75380 msgr2=0x556226b79e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b75380 0x556226b79e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 -- 172.21.15.96:0/802177857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 msgr2=0x556226b7c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b7c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 -- 172.21.15.96:0/802177857 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562268b57e0 con 0x556226b74990 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f022e59c640 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0220099230 tx=0x7f022000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7a07bc793e0b93 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f0224ff9640 1 -- 172.21.15.96:0/802177857 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022009c860 con 0x556226b74990 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f0224ff9640 1 -- 172.21.15.96:0/802177857 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200a09b0 con 0x556226b74990 2024-03-18T15:41:19.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.776+0000 7f0224ff9640 1 -- 172.21.15.96:0/802177857 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220003780 con 0x556226b74990 2024-03-18T15:41:19.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 -- 172.21.15.96:0/802177857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 msgr2=0x556226b74db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0220099230 tx=0x7f022000cf50 comp rx=0 tx=0).stop 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 -- 172.21.15.96:0/802177857 shutdown_connections 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b7c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b75380 0x556226b79e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 --2- 172.21.15.96:0/802177857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b74db0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 -- 172.21.15.96:0/802177857 >> 172.21.15.96:0/802177857 conn(0x556226b3cc50 msgr2=0x556226b70300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 -- 172.21.15.96:0/802177857 shutdown_connections 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.777+0000 7f023743c740 1 -- 172.21.15.96:0/802177857 wait complete. 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 -- start start 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 Processor -- start 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b7f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556226b7d4a0 con 0x556226b7a4f0 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556226a950d0 con 0x556226b75380 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556226b7d620 con 0x556226b74990 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f023743c740 10 client.0 _ll_register_callbacks cb 0x556226a5e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f022ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f022ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53156/0 (socket says 172.21.15.96:53156) 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f022e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f022ed9d640 1 -- 172.21.15.96:0/2734470609 learned_addr learned my addr 172.21.15.96:0/2734470609 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.778+0000 7f0227fff640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b7f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 -- 172.21.15.96:0/2734470609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 msgr2=0x556226b7f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b7f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 -- 172.21.15.96:0/2734470609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 msgr2=0x556226b86460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 -- 172.21.15.96:0/2734470609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556226915f70 con 0x556226b74990 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022ed9d640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0227fff640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b7f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f022e59c640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7eee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f0220009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66c1aa17ce06e2f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200b9060 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02200a09b0 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0220003780 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f022009c860 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f023743c740 10 client.25720 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562267ad170 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.779+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200a09b0 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f022009db90 con 0x556226b74990 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 10 client.25720 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 10 client.25720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 20 client.25720 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226b8a5c0 0x556226b8c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 4 client.25720 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556226ae6610 con 0x556226b8a5c0 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f0227fff640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226b8a5c0 0x556226b8c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f0215ffb640 10 client.25720 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.780+0000 7f0227fff640 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226b8a5c0 0x556226b8c9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbf12c673d97048 server_cookie=8b54949d0e81a6d8 in_seq=0 out_seq=0 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f0215ffb640 1 -- 172.21.15.96:0/2734470609 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556226ae6610 con 0x556226b8a5c0 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f0215ffb640 10 client.25720 handle_command_reply: tid=0 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 client.25720 shutdown 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 msgr2=0x556226b7eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7eee0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0220005fd0 tx=0x7f0220009820 comp rx=0 tx=0).stop 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 shutdown_connections 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226b8a5c0 0x556226b8c9b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b7a4f0 0x556226b86460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f0215ffb640 0 client.25720 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b7f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 --2- 172.21.15.96:0/2734470609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7eee0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 >> 172.21.15.96:0/2734470609 conn(0x556226b3cc50 msgr2=0x556226b6f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.781+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 shutdown_connections 2024-03-18T15:41:19.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.782+0000 7f023743c740 1 -- 172.21.15.96:0/2734470609 wait complete. 2024-03-18T15:41:19.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.782+0000 7f023743c740 20 client.25720 trim_cache size 0 max 16384 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 Processor -- start 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 -- start start 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a6bf0 con 0x556226b75380 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a87f0 con 0x556226b7a4f0 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.817+0000 7f023743c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a72c0 con 0x556226b74990 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48118/0 (socket says 172.21.15.96:48118) 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53166/0 (socket says 172.21.15.96:53166) 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60478/0 (socket says 172.21.15.96:60478) 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022e59c640 1 -- 172.21.15.96:0/4177067288 learned_addr learned my addr 172.21.15.96:0/4177067288 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 -- 172.21.15.96:0/4177067288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 msgr2=0x556226b7f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 -- 172.21.15.96:0/4177067288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 msgr2=0x556226b6f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 -- 172.21.15.96:0/4177067288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556226b8cf40 con 0x556226b75380 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022ed9d640 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f022e59c640 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0227fff640 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f021802c180 tx=0x7f021802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efd0beb123f666d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0217fff640 1 -- 172.21.15.96:0/4177067288 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021805a070 con 0x556226b75380 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0217fff640 1 -- 172.21.15.96:0/4177067288 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02180044c0 con 0x556226b75380 2024-03-18T15:41:19.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.818+0000 7f0217fff640 1 -- 172.21.15.96:0/4177067288 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0218039040 con 0x556226b75380 2024-03-18T15:41:19.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 -- 172.21.15.96:0/4177067288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 msgr2=0x556226b863f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f021802c180 tx=0x7f021802e000 comp rx=0 tx=0).stop 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 -- 172.21.15.96:0/4177067288 shutdown_connections 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b7a4f0 0x556226b6f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b75380 0x556226b863f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 --2- 172.21.15.96:0/4177067288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b74990 0x556226b7f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 -- 172.21.15.96:0/4177067288 >> 172.21.15.96:0/4177067288 conn(0x556226b3cc50 msgr2=0x556226b6ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.819+0000 7f023743c740 1 -- 172.21.15.96:0/4177067288 shutdown_connections 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 -- 172.21.15.96:0/4177067288 wait complete. 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 -- start start 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 Processor -- start 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b84060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b7a4f0 0x556226b85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a8060 con 0x556226b74990 2024-03-18T15:41:19.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a7990 con 0x556226b75380 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f02301a4bd0 con 0x556226b7a4f0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f023743c740 10 client.0 _ll_register_callbacks cb 0x7f0228009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f022e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b84060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f022e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b84060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53180/0 (socket says 172.21.15.96:53180) 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f022e59c640 1 -- 172.21.15.96:0/1482486119 learned_addr learned my addr 172.21.15.96:0/1482486119 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f022ed9d640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b7a4f0 0x556226b85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.820+0000 7f0227fff640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 -- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b7a4f0 msgr2=0x556226b85560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b7a4f0 0x556226b85560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 -- 172.21.15.96:0/1482486119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 msgr2=0x556226b84060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b84060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 -- 172.21.15.96:0/1482486119 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556226b85b50 con 0x556226b75380 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0227fff640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b845a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0218009830 tx=0x7f0218005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53aa2725c1391f67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:19.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021805a070 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f021802c960 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556226ae0960 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0218039050 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f023743c740 10 client.25676 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.821+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556226bf0db0 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556226ae0960 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f02180035f0 con 0x556226b75380 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 10 client.25676 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 10 client.25676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 20 client.25676 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226bf17f0 0x556226bf3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 4 client.25676 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556226a5ebe0 con 0x556226bf17f0 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.822+0000 7f022e59c640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226bf17f0 0x556226bf3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.823+0000 7f0215ffb640 10 client.25676 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.823+0000 7f022e59c640 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226bf17f0 0x556226bf3c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa0b651a1c37b75 server_cookie=c690b0917030daba in_seq=0 out_seq=0 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.823+0000 7f0215ffb640 1 -- 172.21.15.96:0/1482486119 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556226a5ebe0 con 0x556226bf17f0 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.823+0000 7f0215ffb640 10 client.25676 handle_command_reply: tid=0 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.823+0000 7f023743c740 1 client.25676 shutdown 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 msgr2=0x556226b845a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b845a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0218009830 tx=0x7f0218005f10 comp rx=0 tx=0).stop 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 shutdown_connections 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556226b7a4f0 0x556226b85560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556226b75380 0x556226b845a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556226bf17f0 0x556226bf3c10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 --2- 172.21.15.96:0/1482486119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556226b74990 0x556226b84060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f0215ffb640 0 client.25676 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 >> 172.21.15.96:0/1482486119 conn(0x556226b3cc50 msgr2=0x556226b7f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 shutdown_connections 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 1 -- 172.21.15.96:0/1482486119 wait complete. 2024-03-18T15:41:19.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023743c740 20 client.25676 trim_cache size 0 max 16384 2024-03-18T15:41:19.830 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:41:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "049ece28-4656-4cb9-9c10-3ca83155f21d", 2024-03-18T15:41:19.830 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:41:19.830 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 msgr2=0x7f021c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 0x7f021c0a4d50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0230199010 tx=0x7f0218034ae0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 msgr2=0x7f0230197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 0x7f0230197b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0230068740 tx=0x7f022000ee50 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 shutdown_connections 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f021c0a2890 0x7f021c0a4d50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0230103430 0x7f023019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.824+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0230102a70 0x7f0230198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.825+0000 7f023484b640 1 --2- 172.21.15.96:0/536927584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02300ffaa0 0x7f0230197b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.825+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 >> 172.21.15.96:0/536927584 conn(0x7f02300fb3d0 msgr2=0x7f0230105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.825+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 shutdown_connections 2024-03-18T15:41:19.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.825+0000 7f023484b640 1 -- 172.21.15.96:0/536927584 wait complete. 2024-03-18T15:41:19.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:41:19.842 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 -- 172.21.15.96:0/4266115256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 msgr2=0x7f9b001082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 --2- 172.21.15.96:0/4266115256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b001082b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009be0 tx=0x7f9af802bef0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 -- 172.21.15.96:0/4266115256 shutdown_connections 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 --2- 172.21.15.96:0/4266115256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b001082b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 --2- 172.21.15.96:0/4266115256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 --2- 172.21.15.96:0/4266115256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b001002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.861+0000 7f9b076e8640 1 -- 172.21.15.96:0/4266115256 >> 172.21.15.96:0/4266115256 conn(0x7f9b000fb430 msgr2=0x7f9b000fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.862+0000 7f9b076e8640 1 -- 172.21.15.96:0/4266115256 shutdown_connections 2024-03-18T15:41:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.862+0000 7f9b076e8640 1 -- 172.21.15.96:0/4266115256 wait complete. 2024-03-18T15:41:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 Processor -- start 2024-03-18T15:41:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 -- start start 2024-03-18T15:41:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b0019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0010c400 con 0x7f9b00100fd0 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0010c280 con 0x7f9b00101990 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48148/0 (socket says 172.21.15.96:48148) 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b05484640 1 -- 172.21.15.96:0/1044072375 learned_addr learned my addr 172.21.15.96:0/1044072375 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b04c83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b0019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.863+0000 7f9b076e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0010c580 con 0x7f9b000ffe80 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 -- 172.21.15.96:0/1044072375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 msgr2=0x7f9b00196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 -- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 msgr2=0x7f9b00196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05c85640 1 -- 172.21.15.96:0/1044072375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b00108da0 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b05484640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.864+0000 7f9b04c83640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9b05c85640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b0019ded0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9af802c430 tx=0x7f9af80337b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9af67fc640 1 -- 172.21.15.96:0/1044072375 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8043020 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9af67fc640 1 -- 172.21.15.96:0/1044072375 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af802ca60 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b00109030 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9af67fc640 1 -- 172.21.15.96:0/1044072375 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8034690 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.865+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001095d0 con 0x7f9b00101990 2024-03-18T15:41:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.867+0000 7f9af67fc640 1 -- 172.21.15.96:0/1044072375 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f9af8046020 con 0x7f9b00101990 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9af67fc640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 0x7f9aec0a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 Processor -- start 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 -- start start 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9af67fc640 1 -- 172.21.15.96:0/1044072375 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9af80f0fc0 con 0x7f9b00101990 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b05484640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 0x7f9aec0a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291284830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291287330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632911e4a40 con 0x56329127f340 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56329118d920 con 0x56329127fd30 2024-03-18T15:41:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632911e0ae0 con 0x563291284ea0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b05484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b05484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53208/0 (socket says 172.21.15.96:53208) 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b05484640 1 -- 172.21.15.96:0/2955767327 learned_addr learned my addr 172.21.15.96:0/2955767327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b04c83640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291284830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.868+0000 7f9b05c85640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291287330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 0x7f9aec0a4e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009c00 tx=0x7f9afc00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 -- 172.21.15.96:0/2955767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 msgr2=0x563291287330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291287330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 -- 172.21.15.96:0/2955767327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 msgr2=0x563291284830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291284830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 -- 172.21.15.96:0/2955767327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632911a5460 con 0x56329127f340 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b04c83640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291284830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9b05484640 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b7c0 tx=0x7f9afc024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bfd337d28ac947 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9aeb7fe640 1 -- 172.21.15.96:0/2955767327 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00be80 con 0x56329127f340 2024-03-18T15:41:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9aeb7fe640 1 -- 172.21.15.96:0/2955767327 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc02a8f0 con 0x56329127f340 2024-03-18T15:41:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.869+0000 7f9aeb7fe640 1 -- 172.21.15.96:0/2955767327 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc029880 con 0x56329127f340 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2955767327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 msgr2=0x56329127f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b7c0 tx=0x7f9afc024000 comp rx=0 tx=0).stop 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2955767327 shutdown_connections 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291287330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291284830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2955767327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x56329127f760 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.870+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2955767327 >> 172.21.15.96:0/2955767327 conn(0x56329127a8b0 msgr2=0x56329127acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.871+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2955767327 shutdown_connections 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.871+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2955767327 wait complete. 2024-03-18T15:41:19.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.871+0000 7f9b0a3ac740 1 -- start start 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.871+0000 7f9b0a3ac740 1 Processor -- start 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291289b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b05484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b05484640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53210/0 (socket says 172.21.15.96:53210) 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b05484640 1 -- 172.21.15.96:0/4245154085 learned_addr learned my addr 172.21.15.96:0/4245154085 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291290b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563291287d90 con 0x56329127f340 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5632911a0d70 con 0x56329127fd30 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563291287f10 con 0x563291284ea0 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b04c83640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291289b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b05c85640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291290b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.872+0000 7f9b0a3ac740 10 client.0 _ll_register_callbacks cb 0x563290ef8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:19.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 -- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 msgr2=0x563291290b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291290b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 -- 172.21.15.96:0/4245154085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 msgr2=0x563291289b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291289b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563290ef9210 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b05484640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009520 tx=0x7f9afc005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1353dd168060c17b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9ae8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc02aaa0 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9afc02ac40 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9afc02a8f0 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9afc02a410 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b0a3ac740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.873+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563290fc5620 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc02a8f0 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f9afc03c070 con 0x56329127f340 2024-03-18T15:41:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 20 client.15897 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5632912951d0 0x563291297640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 4 client.15897 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632911f1f80 con 0x5632912951d0 2024-03-18T15:41:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.874+0000 7f9b04c83640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5632912951d0 0x563291297640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.875+0000 7f9ae8ff9640 10 client.15897 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.875+0000 7f9b04c83640 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5632912951d0 0x563291297640 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5012399c7c153c server_cookie=48e9c101b156e052 in_seq=0 out_seq=0 2024-03-18T15:41:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.875+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/4245154085 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632911f1f80 con 0x5632912951d0 2024-03-18T15:41:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.875+0000 7f9ae8ff9640 10 client.15897 handle_command_reply: tid=0 2024-03-18T15:41:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.875+0000 7f9b0a3ac740 1 client.15897 shutdown 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 msgr2=0x5632912895e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009520 tx=0x7f9afc005df0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 shutdown_connections 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563291284ea0 0x563291290b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127fd30 0x563291289b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5632912951d0 0x563291297640 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/4245154085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127f340 0x5632912895e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 >> 172.21.15.96:0/4245154085 conn(0x56329127a8b0 msgr2=0x563291279bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9ae8ff9640 0 client.15897 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 shutdown_connections 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/4245154085 wait complete. 2024-03-18T15:41:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.876+0000 7f9b0a3ac740 20 client.15897 trim_cache size 0 max 16384 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 Processor -- start 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 -- start start 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x5632912891f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x563291279320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a6ab0 con 0x56329127fd30 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a86b0 con 0x56329127f340 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a7180 con 0x56329127a8b0 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b04c83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x5632912891f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b04c83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60520/0 (socket says 172.21.15.96:60520) 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x5632912891f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48176/0 (socket says 172.21.15.96:48176) 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b04c83640 1 -- 172.21.15.96:0/2451492571 learned_addr learned my addr 172.21.15.96:0/2451492571 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.907+0000 7f9b05c85640 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x563291279320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 -- 172.21.15.96:0/2451492571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 msgr2=0x5632912891f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x5632912891f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 -- 172.21.15.96:0/2451492571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 msgr2=0x563291279320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x563291279320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 -- 172.21.15.96:0/2451492571 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329127f720 con 0x56329127f340 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9b04c83640 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5632912807a0 tx=0x7f9af0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb0a34f84a6315 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9aeaffd640 1 -- 172.21.15.96:0/2451492571 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0026b20 con 0x56329127f340 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9aeaffd640 1 -- 172.21.15.96:0/2451492571 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af0026cc0 con 0x56329127f340 2024-03-18T15:41:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.908+0000 7f9aeaffd640 1 -- 172.21.15.96:0/2451492571 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0005280 con 0x56329127f340 2024-03-18T15:41:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2451492571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 msgr2=0x563291289730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5632912807a0 tx=0x7f9af0005f90 comp rx=0 tx=0).stop 2024-03-18T15:41:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2451492571 shutdown_connections 2024-03-18T15:41:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x563291279320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x563291289730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/2451492571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x5632912891f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2451492571 >> 172.21.15.96:0/2451492571 conn(0x5632912849e0 msgr2=0x563291275a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2451492571 shutdown_connections 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.909+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/2451492571 wait complete. 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 -- start start 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 Processor -- start 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x56329128e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x56329128f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a7f20 con 0x56329127fd30 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a7850 con 0x56329127f340 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001a4a90 con 0x56329127a8b0 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48186/0 (socket says 172.21.15.96:48186) 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x56329128f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 -- 172.21.15.96:0/559320139 learned_addr learned my addr 172.21.15.96:0/559320139 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b04c83640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x56329128e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 -- 172.21.15.96:0/559320139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 msgr2=0x56329128e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x56329128e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 -- 172.21.15.96:0/559320139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 msgr2=0x56329128f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x56329128f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 -- 172.21.15.96:0/559320139 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329128f980 con 0x56329127a8b0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b04c83640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x56329128e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.910+0000 7f9b0a3ac740 10 client.0 _ll_register_callbacks cb 0x7f9af0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af0026cc0 con 0x56329127a8b0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b05484640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b7c0 tx=0x7f9afc009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337471bdf5ddd300 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9ae8ff9640 10 client.25744 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/559320139 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc04c050 con 0x56329127a8b0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/559320139 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9afc004890 con 0x56329127a8b0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/559320139 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9afc042070 con 0x56329127a8b0 2024-03-18T15:41:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b0a3ac740 10 client.25744 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.911+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=27}) -- 0x5632912fb540 con 0x56329127a8b0 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/559320139 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f9afc03c040 con 0x56329127a8b0 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 10 client.25744 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 20 client.25744 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563291284ea0 0x5632912fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 4 client.25744 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632911ec2d0 con 0x563291284ea0 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b04c83640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563291284ea0 0x5632912fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9b04c83640 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563291284ea0 0x5632912fdc80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34c3cf4b73d70de server_cookie=663184d261498bc5 in_seq=0 out_seq=0 2024-03-18T15:41:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.912+0000 7f9ae8ff9640 10 client.25744 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.924+0000 7f9ae8ff9640 1 -- 172.21.15.96:0/559320139 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632911ec2d0 con 0x563291284ea0 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.924+0000 7f9ae8ff9640 10 client.25744 handle_command_reply: tid=0 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.924+0000 7f9b0a3ac740 1 client.25744 shutdown 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 msgr2=0x56329128de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b7c0 tx=0x7f9afc009980 comp rx=0 tx=0).stop 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 shutdown_connections 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563291284ea0 0x5632912fdc80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56329127fd30 0x56329128f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9ae8ff9640 0 client.25744 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56329127f340 0x56329128e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 --2- 172.21.15.96:0/559320139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56329127a8b0 0x56329128de40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 >> 172.21.15.96:0/559320139 conn(0x5632912849e0 msgr2=0x563291289160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 shutdown_connections 2024-03-18T15:41:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 1 -- 172.21.15.96:0/559320139 wait complete. 2024-03-18T15:41:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.925+0000 7f9b0a3ac740 20 client.25744 trim_cache size 0 max 16384 2024-03-18T15:41:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:41:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:41:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:41:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 msgr2=0x7f9aec0a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 0x7f9aec0a4e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9afc009c00 tx=0x7f9afc00b2c0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 msgr2=0x7f9b0019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b0019ded0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9af802c430 tx=0x7f9af80337b0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 shutdown_connections 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9aec0a29c0 0x7f9aec0a4e80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101990 0x7f9b0019ded0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b00100fd0 0x7f9b00196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 --2- 172.21.15.96:0/1044072375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b000ffe80 0x7f9b00196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 >> 172.21.15.96:0/1044072375 conn(0x7f9b000fb430 msgr2=0x7f9b00104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 shutdown_connections 2024-03-18T15:41:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:19.926+0000 7f9b076e8640 1 -- 172.21.15.96:0/1044072375 wait complete. 2024-03-18T15:41:19.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 -- 172.21.15.96:0/1806777116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc1033c0 msgr2=0x7fa5bc109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/1806777116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc109c50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009900 tx=0x7fa5ac02bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 -- 172.21.15.96:0/1806777116 shutdown_connections 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/1806777116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc109c50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/1806777116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc102a00 0x7fa5bc102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/1806777116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc101810 0x7fa5bc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 -- 172.21.15.96:0/1806777116 >> 172.21.15.96:0/1806777116 conn(0x7fa5bc0fcd80 msgr2=0x7fa5bc0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 -- 172.21.15.96:0/1806777116 shutdown_connections 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.976+0000 7fa5c35e6640 1 -- 172.21.15.96:0/1806777116 wait complete. 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 Processor -- start 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 -- start start 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc102a00 0x7fa5bc197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc10d490 con 0x7fa5bc101810 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc10d310 con 0x7fa5bc1033c0 2024-03-18T15:41:19.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c35e6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc10d610 con 0x7fa5bc102a00 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53250/0 (socket says 172.21.15.96:53250) 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 -- 172.21.15.96:0/673750888 learned_addr learned my addr 172.21.15.96:0/673750888 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc102a00 0x7fa5bc197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 -- 172.21.15.96:0/673750888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc102a00 msgr2=0x7fa5bc197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc102a00 0x7fa5bc197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 -- 172.21.15.96:0/673750888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 msgr2=0x7fa5bc19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 -- 172.21.15.96:0/673750888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc19f6d0 con 0x7fa5bc101810 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5c1382640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400eb70 tx=0x7fa5b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.977+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/673750888 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003b70 con 0x7fa5bc101810 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.978+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc19f9c0 con 0x7fa5bc101810 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.978+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/673750888 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b400ee70 con 0x7fa5bc101810 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.978+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/673750888 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4010830 con 0x7fa5bc101810 2024-03-18T15:41:19.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.978+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc19fec0 con 0x7fa5bc101810 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.980+0000 7fa5c61d2740 1 Processor -- start 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.980+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/673750888 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7fa5b4004020 con 0x7fa5bc101810 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.980+0000 7fa5c61d2740 1 -- start start 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 0x557b22c13660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b22b73500 con 0x557b22c0eb60 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b22b1bbe0 con 0x557b22c13cd0 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557b22b6f5a0 con 0x557b22c0e170 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 0x557b22c13660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1b83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1b83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48228/0 (socket says 172.21.15.96:48228) 2024-03-18T15:41:19.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1b83640 1 -- 172.21.15.96:0/1455383582 learned_addr learned my addr 172.21.15.96:0/1455383582 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 -- 172.21.15.96:0/1455383582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 msgr2=0x557b22c0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 -- 172.21.15.96:0/1455383582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 msgr2=0x557b22c16160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c16160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 -- 172.21.15.96:0/1455383582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b22b2f4b0 con 0x557b22c0eb60 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c16160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5ba7fc640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 0x7fa5a40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 0x557b22c13660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40cccc0 tx=0x7fa5b40d74d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1348bd6fce5c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.981+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 0x7fa5a40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5a37fe640 1 -- 172.21.15.96:0/1455383582 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4097df0 con 0x557b22c0eb60 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5a37fe640 1 -- 172.21.15.96:0/1455383582 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4022000 con 0x557b22c0eb60 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5a37fe640 1 -- 172.21.15.96:0/1455383582 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40c5ac0 con 0x557b22c0eb60 2024-03-18T15:41:19.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5ba7fc640 1 -- 172.21.15.96:0/673750888 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa5b40cf340 con 0x7fa5bc101810 2024-03-18T15:41:19.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 0x7fa5a40a4d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005c40 tx=0x7fa5ac005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.982+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1455383582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 msgr2=0x557b22c13660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 0x557b22c13660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40cccc0 tx=0x7fa5b40d74d0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1455383582 shutdown_connections 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c16160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0eb60 0x557b22c13660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1455383582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0e170 0x557b22c0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1455383582 >> 172.21.15.96:0/1455383582 conn(0x557b22c096e0 msgr2=0x557b22c09ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1455383582 shutdown_connections 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1455383582 wait complete. 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- start start 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 Processor -- start 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b22c16aa0 con 0x557b22c0e170 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b22b2f160 con 0x557b22c13cd0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557b22c16c20 con 0x557b22c0eb60 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.983+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c61d2740 10 client.0 _ll_register_callbacks cb 0x557b22b13150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53268/0 (socket says 172.21.15.96:53268) 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 -- 172.21.15.96:0/1323805455 learned_addr learned my addr 172.21.15.96:0/1323805455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 -- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 msgr2=0x557b22c189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 -- 172.21.15.96:0/1323805455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 msgr2=0x557b22c1f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 -- 172.21.15.96:0/1323805455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b229517d0 con 0x557b22c0e170 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa5b409b850 tx=0x7fa5b40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa70469ab8cf56a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:19.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40c35d0 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa5b40ccf10 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b22b80a40 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5b40c37a0 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c61d2740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b228873f0 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.984+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557b22b80a40 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b4004310 con 0x557b22c0e170 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 20 client.15912 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c23c60 0x557b22c260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 4 client.15912 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b22b7ad90 con 0x557b22c23c60 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c23c60 0x557b22c260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:19.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c23c60 0x557b22c260c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be846cb655eba0de server_cookie=1657a68b4f8fdf65 in_seq=0 out_seq=0 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.985+0000 7fa5a0ff9640 10 client.15912 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/1323805455 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557b22b7ad90 con 0x557b22c23c60 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5a0ff9640 10 client.15912 handle_command_reply: tid=0 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 client.15912 shutdown 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 msgr2=0x557b22c18470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa5b409b850 tx=0x7fa5b40021d0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 shutdown_connections 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c23c60 0x557b22c260c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1323805455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c18470 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5a0ff9640 0 client.15912 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.986+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 >> 172.21.15.96:0/1323805455 conn(0x557b22c096e0 msgr2=0x557b22c08900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.987+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 shutdown_connections 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.987+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1323805455 wait complete. 2024-03-18T15:41:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:19.987+0000 7fa5c61d2740 20 client.15912 trim_cache size 0 max 16384 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 Processor -- start 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 -- start start 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1a8d90 con 0x557b22c13cd0 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1aa990 con 0x557b22c0e170 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1a9460 con 0x557b22c0eb60 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c1b83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60588/0 (socket says 172.21.15.96:60588) 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c1b83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53282/0 (socket says 172.21.15.96:53282) 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c1382640 1 -- 172.21.15.96:0/1750597806 learned_addr learned my addr 172.21.15.96:0/1750597806 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.004+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 -- 172.21.15.96:0/1750597806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 msgr2=0x557b22c1f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 -- 172.21.15.96:0/1750597806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 msgr2=0x557b22c17fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 -- 172.21.15.96:0/1750597806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b228470f0 con 0x557b22c0e170 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5c1382640 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4097fb0 tx=0x7fa5b4004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a698c3f8547345c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5a2ffd640 1 -- 172.21.15.96:0/1750597806 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40c1080 con 0x557b22c0e170 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5a2ffd640 1 -- 172.21.15.96:0/1750597806 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b409c250 con 0x557b22c0e170 2024-03-18T15:41:20.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.005+0000 7fa5a2ffd640 1 -- 172.21.15.96:0/1750597806 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b409b4b0 con 0x557b22c0e170 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1750597806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 msgr2=0x557b22c17a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4097fb0 tx=0x7fa5b4004680 comp rx=0 tx=0).stop 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1750597806 shutdown_connections 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c13cd0 0x557b22c17fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/1750597806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c0e170 0x557b22c17a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1750597806 >> 172.21.15.96:0/1750597806 conn(0x557b22c096e0 msgr2=0x557b22c0d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1750597806 shutdown_connections 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- 172.21.15.96:0/1750597806 wait complete. 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- start start 2024-03-18T15:41:20.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 Processor -- start 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1aa200 con 0x557b22c0e170 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1a9b30 con 0x557b22c13cd0 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.006+0000 7fa5c61d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1a6d70 con 0x557b22c0eb60 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c61d2740 10 client.0 _ll_register_callbacks cb 0x7fa5b00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53284/0 (socket says 172.21.15.96:53284) 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 -- 172.21.15.96:0/696048519 learned_addr learned my addr 172.21.15.96:0/696048519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1b83640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 -- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 msgr2=0x557b22c1d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 -- 172.21.15.96:0/696048519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 msgr2=0x557b22c1e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 -- 172.21.15.96:0/696048519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b22b3a9d0 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c1382640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa5b409c400 tx=0x7fa5b40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa00ba4783ae4ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40ba060 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa5b4012570 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b409c250 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5b409b690 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c61d2740 10 client.15918 fetch_fsmap learned FSMap version 27 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557b22b3a9d0 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.007+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b409c250 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b40ba200 con 0x557b22c0e170 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:20.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 20 client.15918 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c89fb0 0x557b22c8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 4 client.15918 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b22af8540 con 0x557b22c89fb0 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c89fb0 0x557b22c8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5a0ff9640 10 client.15918 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c0b81640 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c89fb0 0x557b22c8c3a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1121182ce5321a server_cookie=b328090b463644c9 in_seq=0 out_seq=0 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5a0ff9640 1 -- 172.21.15.96:0/696048519 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x557b22af8540 con 0x557b22c89fb0 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5a0ff9640 10 client.15918 handle_command_reply: tid=0 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.008+0000 7fa5c61d2740 1 client.15918 shutdown 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 msgr2=0x557b22c1cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa5b409c400 tx=0x7fa5b40046f0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 shutdown_connections 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b22c13cd0 0x557b22c1e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b22c0eb60 0x557b22c1d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557b22c89fb0 0x557b22c8c3a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 --2- 172.21.15.96:0/696048519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b22c0e170 0x557b22c1cc20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5a0ff9640 0 client.15918 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 >> 172.21.15.96:0/696048519 conn(0x557b22c096e0 msgr2=0x557b22c08070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 shutdown_connections 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 1 -- 172.21.15.96:0/696048519 wait complete. 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c61d2740 20 client.15918 trim_cache size 0 max 16384 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "049ece28-4656-4cb9-9c10-3ca83155f21d": { 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "049ece28-4656-4cb9-9c10-3ca83155f21d", 2024-03-18T15:41:20.016 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:41:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:41:20.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:41:20.017 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:20.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 msgr2=0x7fa5a40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 0x7fa5a40a4d50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005c40 tx=0x7fa5ac005a20 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 msgr2=0x7fa5bc197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400eb70 tx=0x7fa5b400c6a0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 shutdown_connections 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a40a2890 0x7fa5a40a4d50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5bc1033c0 0x7fa5bc19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5bc102a00 0x7fa5bc197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 --2- 172.21.15.96:0/673750888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5bc101810 0x7fa5bc197ab0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 >> 172.21.15.96:0/673750888 conn(0x7fa5bc0fcd80 msgr2=0x7fa5bc105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 shutdown_connections 2024-03-18T15:41:20.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:20.009+0000 7fa5c35e6640 1 -- 172.21.15.96:0/673750888 wait complete. 2024-03-18T15:41:20.020 INFO:tasks.cephfs.filesystem:scrub status for tag:049ece28-4656-4cb9-9c10-3ca83155f21d - {'path': '/', 'tag': '049ece28-4656-4cb9-9c10-3ca83155f21d', 'options': 'recursive,force'} 2024-03-18T15:41:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:20 smithi193 ceph-mon[36702]: pgmap v389: 97 pgs: 97 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-03-18T15:41:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:20 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:41:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:20 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:20 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:20 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:20 smithi096 ceph-mon[31930]: pgmap v389: 97 pgs: 97 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-03-18T15:41:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:20 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:41:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:20 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:20 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:20 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:20 smithi137 ceph-mon[35868]: pgmap v389: 97 pgs: 97 active+clean; 35 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 139 op/s 2024-03-18T15:41:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:20 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:41:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:20 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:20 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:20 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:21.327 INFO:tasks.workunit.client.0.smithi096.stdout: 15 142730 685 97922 696 45296 724 2024-03-18T15:41:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:21 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:21.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:21 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:22.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:21 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:22 smithi193 ceph-mon[36702]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.3 MiB/s wr, 165 op/s 2024-03-18T15:41:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:22 smithi096 ceph-mon[31930]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.3 MiB/s wr, 165 op/s 2024-03-18T15:41:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:22 smithi137 ceph-mon[35868]: pgmap v390: 97 pgs: 97 active+clean; 41 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 3.3 MiB/s wr, 165 op/s 2024-03-18T15:41:24.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:24 smithi137 ceph-mon[35868]: pgmap v391: 97 pgs: 97 active+clean; 43 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.6 MiB/s wr, 186 op/s 2024-03-18T15:41:24.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:24 smithi193 ceph-mon[36702]: pgmap v391: 97 pgs: 97 active+clean; 43 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.6 MiB/s wr, 186 op/s 2024-03-18T15:41:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:24 smithi096 ceph-mon[31930]: pgmap v391: 97 pgs: 97 active+clean; 43 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 3.6 MiB/s wr, 186 op/s 2024-03-18T15:41:25.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:25 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:25.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:25 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:26.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:25 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:26.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:26 smithi193 ceph-mon[36702]: pgmap v392: 97 pgs: 97 active+clean; 49 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 191 op/s 2024-03-18T15:41:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:26 smithi096 ceph-mon[31930]: pgmap v392: 97 pgs: 97 active+clean; 49 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 191 op/s 2024-03-18T15:41:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:26 smithi137 ceph-mon[35868]: pgmap v392: 97 pgs: 97 active+clean; 49 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 191 op/s 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.922 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.923 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.924 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.925 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.926 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.927 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.928 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.929 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.930 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.931 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:27 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.938 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.939 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.940 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.943 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.944 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.945 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.946 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:27.947 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:27 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.109 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.111 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.112 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.113 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.114 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.115 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.116 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.117 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.118 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:27 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:28 smithi193 ceph-mon[36702]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 190 op/s 2024-03-18T15:41:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:41:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:41:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:28 smithi096 ceph-mon[31930]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 190 op/s 2024-03-18T15:41:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:41:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:41:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:28 smithi137 ceph-mon[35868]: pgmap v393: 97 pgs: 97 active+clean; 49 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 190 op/s 2024-03-18T15:41:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:41:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:41:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:41:30.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:30 smithi193 ceph-mon[36702]: pgmap v394: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-03-18T15:41:30.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:30 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:30 smithi096 ceph-mon[31930]: pgmap v394: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-03-18T15:41:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:30 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:30 smithi137 ceph-mon[35868]: pgmap v394: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 171 op/s 2024-03-18T15:41:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:30 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:41:31.328 INFO:tasks.workunit.client.0.smithi096.stdout: 23 346537 508 243198 431 108194 808 2024-03-18T15:41:31.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:31 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:31.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:31 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:32.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:31 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:32 smithi193 ceph-mon[36702]: pgmap v395: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-03-18T15:41:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:32 smithi096 ceph-mon[31930]: pgmap v395: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 -- 172.21.15.96:0/963128121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 msgr2=0x7f3118108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 --2- 172.21.15.96:0/963128121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f3118108b40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3110009ca0 tx=0x7f311002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 -- 172.21.15.96:0/963128121 shutdown_connections 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 --2- 172.21.15.96:0/963128121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f3118108b40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 --2- 172.21.15.96:0/963128121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31181018f0 0x7f3118101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 --2- 172.21.15.96:0/963128121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3118100700 0x7f3118100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 -- 172.21.15.96:0/963128121 >> 172.21.15.96:0/963128121 conn(0x7f31180fbc70 msgr2=0x7f31180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 -- 172.21.15.96:0/963128121 shutdown_connections 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.015+0000 7f311d6bd640 1 -- 172.21.15.96:0/963128121 wait complete. 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 Processor -- start 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 -- start start 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3118100700 0x7f3118196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31181018f0 0x7f3118196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f311810c380 con 0x7f31181022b0 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f311810c200 con 0x7f31181018f0 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f311d6bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f311810c500 con 0x7f3118100700 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f31177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f31177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38942/0 (socket says 172.21.15.96:38942) 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f31177fe640 1 -- 172.21.15.96:0/1871839965 learned_addr learned my addr 172.21.15.96:0/1871839965 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:33.019 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f3116ffd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3118100700 0x7f3118196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.016+0000 7f31167fc640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31181018f0 0x7f3118196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 -- 172.21.15.96:0/1871839965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3118100700 msgr2=0x7f3118196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3118100700 0x7f3118196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 -- 172.21.15.96:0/1871839965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31181018f0 msgr2=0x7f3118196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31181018f0 0x7f3118196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 -- 172.21.15.96:0/1871839965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311819e550 con 0x7f31181022b0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f31177fe640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3110037040 tx=0x7f3110032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f30fffff640 1 -- 172.21.15.96:0/1871839965 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110041020 con 0x7f31181022b0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311819e7e0 con 0x7f31181022b0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f30fffff640 1 -- 172.21.15.96:0/1871839965 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3110034cd0 con 0x7f31181022b0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f30fffff640 1 -- 172.21.15.96:0/1871839965 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110039040 con 0x7f31181022b0 2024-03-18T15:41:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.017+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311819ed20 con 0x7f31181022b0 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f30fffff640 1 -- 172.21.15.96:0/1871839965 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 137341+0+0 (secure 0 0 0) 0x7f3110044020 con 0x7f31181022b0 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 Processor -- start 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 -- start start 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb620970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6260b0 0x5642eb628540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642eb583a40 con 0x5642eb620f40 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642eb52c920 con 0x5642eb620550 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3120383740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5642eb57fae0 con 0x5642eb6260b0 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38952/0 (socket says 172.21.15.96:38952) 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f3116ffd640 1 -- 172.21.15.96:0/1883540355 learned_addr learned my addr 172.21.15.96:0/1883540355 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f31167fc640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb620970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f31167fc640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6260b0 0x5642eb628540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.020+0000 7f30fffff640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 0x7f31000a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 -- 172.21.15.96:0/1883540355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6260b0 msgr2=0x5642eb628540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f31177fe640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 0x7f31000a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6260b0 0x5642eb628540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 -- 172.21.15.96:0/1883540355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 msgr2=0x5642eb620970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb620970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 -- 172.21.15.96:0/1883540355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642eb544460 con 0x5642eb620f40 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f31167fc640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb620970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f30fffff640 1 -- 172.21.15.96:0/1871839965 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f31100f09a0 con 0x7f31181022b0 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f3116ffd640 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3108009ca0 tx=0x7f3108011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5013644991edf27b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f30fd7fa640 1 -- 172.21.15.96:0/1883540355 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108004220 con 0x5642eb620f40 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f30fd7fa640 1 -- 172.21.15.96:0/1883540355 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31080043c0 con 0x5642eb620f40 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.021+0000 7f30fd7fa640 1 -- 172.21.15.96:0/1883540355 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108019870 con 0x5642eb620f40 2024-03-18T15:41:33.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- 172.21.15.96:0/1883540355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 msgr2=0x5642eb625a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f31177fe640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 0x7f31000a4e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3110090050 tx=0x7f3110009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3108009ca0 tx=0x7f3108011810 comp rx=0 tx=0).stop 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- 172.21.15.96:0/1883540355 shutdown_connections 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6260b0 0x5642eb628540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb625a40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- 172.21.15.96:0/1883540355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb620970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- 172.21.15.96:0/1883540355 >> 172.21.15.96:0/1883540355 conn(0x5642eb6198b0 msgr2=0x5642eb619cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- 172.21.15.96:0/1883540355 shutdown_connections 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- 172.21.15.96:0/1883540355 wait complete. 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- start start 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 Processor -- start 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb62a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 0x5642eb62ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 0x5642eb631df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5642eb629060 con 0x5642eb6260b0 2024-03-18T15:41:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642eb53fd70 con 0x5642eb620f40 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5642eb6291e0 con 0x5642eb620550 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3120383740 10 client.0 _ll_register_callbacks cb 0x5642eb297a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f31167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 0x5642eb631df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 0x5642eb62ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f31167fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb62a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3116ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 0x5642eb62ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:35840/0 (socket says 172.21.15.96:35840) 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.022+0000 7f3116ffd640 1 -- 172.21.15.96:0/3307782587 learned_addr learned my addr 172.21.15.96:0/3307782587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 -- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 msgr2=0x5642eb62a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb62a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 -- 172.21.15.96:0/3307782587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 msgr2=0x5642eb62ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 0x5642eb62ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 -- 172.21.15.96:0/3307782587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642eb298210 con 0x5642eb6260b0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f31167fc640 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 0x5642eb631df0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5642eb6217a0 tx=0x7f310c009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfd13e3185588a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c01a040 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f310c011050 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31080043c0 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f310c011e50 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 10 client.15930 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642eb364620 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31080043c0 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f310c0049a0 con 0x5642eb6260b0 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 10 client.15930 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 20 client.15930 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb636740 0x5642eb638bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 4 client.15930 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.023+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642eb590f80 con 0x5642eb636740 2024-03-18T15:41:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.024+0000 7f3116ffd640 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb636740 0x5642eb638bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.024+0000 7f30faff5640 10 client.15930 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.024+0000 7f3116ffd640 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb636740 0x5642eb638bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdc2daba108104e server_cookie=8879aa94b577a3f5 in_seq=0 out_seq=0 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f30faff5640 1 -- 172.21.15.96:0/3307782587 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5642eb590f80 con 0x5642eb636740 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f30faff5640 10 client.15930 handle_command_reply: tid=0 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 client.15930 shutdown 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 msgr2=0x5642eb631df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 0x5642eb631df0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x5642eb6217a0 tx=0x7f310c009ac0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 shutdown_connections 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb636740 0x5642eb638bc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb6260b0 0x5642eb631df0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620f40 0x5642eb62ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 --2- 172.21.15.96:0/3307782587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb62a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 >> 172.21.15.96:0/3307782587 conn(0x5642eb6198b0 msgr2=0x5642eb618bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f30faff5640 0 client.15930 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 shutdown_connections 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 1 -- 172.21.15.96:0/3307782587 wait complete. 2024-03-18T15:41:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.025+0000 7f3120383740 20 client.15930 trim_cache size 0 max 16384 2024-03-18T15:41:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 Processor -- start 2024-03-18T15:41:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 -- start start 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6198b0 0x5642eb614a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb631db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a6b30 con 0x5642eb620f40 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a8790 con 0x5642eb620550 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.040+0000 7f3120383740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a7200 con 0x5642eb6198b0 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38974/0 (socket says 172.21.15.96:38974) 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 -- 172.21.15.96:0/1911356836 learned_addr learned my addr 172.21.15.96:0/1911356836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6198b0 0x5642eb614a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f3116ffd640 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb631db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 -- 172.21.15.96:0/1911356836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6198b0 msgr2=0x5642eb614a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6198b0 0x5642eb614a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 -- 172.21.15.96:0/1911356836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 msgr2=0x5642eb631db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb631db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 -- 172.21.15.96:0/1911356836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642eb621320 con 0x5642eb620f40 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f31167fc640 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f310c009c30 tx=0x7f310c011400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965246ce0e064748 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f30fcff9640 1 -- 172.21.15.96:0/1911356836 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c02e070 con 0x5642eb620f40 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f30fcff9640 1 -- 172.21.15.96:0/1911356836 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c011e50 con 0x5642eb620f40 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.041+0000 7f30fcff9640 1 -- 172.21.15.96:0/1911356836 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c02b460 con 0x5642eb620f40 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- 172.21.15.96:0/1911356836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 msgr2=0x5642eb618320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f310c009c30 tx=0x7f310c011400 comp rx=0 tx=0).stop 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- 172.21.15.96:0/1911356836 shutdown_connections 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb618320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb620550 0x5642eb631db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- 172.21.15.96:0/1911356836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb6198b0 0x5642eb614a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- 172.21.15.96:0/1911356836 >> 172.21.15.96:0/1911356836 conn(0x5642eb621b40 msgr2=0x5642eb622260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- 172.21.15.96:0/1911356836 shutdown_connections 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- 172.21.15.96:0/1911356836 wait complete. 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- start start 2024-03-18T15:41:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 Processor -- start 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb6198b0 0x5642eb62fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb630020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a8000 con 0x5642eb620f40 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a7930 con 0x5642eb6198b0 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a4b10 con 0x5642eb620550 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f3120383740 10 client.0 _ll_register_callbacks cb 0x7f310c011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f31167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f31167fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38978/0 (socket says 172.21.15.96:38978) 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f31167fc640 1 -- 172.21.15.96:0/2207572743 learned_addr learned my addr 172.21.15.96:0/2207572743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.042+0000 7f31167fc640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb6198b0 0x5642eb62fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f3116ffd640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb630020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 -- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 msgr2=0x5642eb630020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb630020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 -- 172.21.15.96:0/2207572743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb6198b0 msgr2=0x5642eb62fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb6198b0 0x5642eb62fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 -- 172.21.15.96:0/2207572743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642eb6315d0 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f31167fc640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f310c009fd0 tx=0x7f310c009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2825e95d019e440 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f30faff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c02e070 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c011e50 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f310c004fd0 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f310c0051d0 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f3120383740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642eb69d0d0 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.043+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c011e50 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f310c047410 con 0x5642eb620f40 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 20 client.15936 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb6260b0 0x5642eb69fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 4 client.15936 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642eb58b2d0 con 0x5642eb6260b0 2024-03-18T15:41:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3116ffd640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb6260b0 0x5642eb69fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f3116ffd640 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb6260b0 0x5642eb69fb50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aebcf9bab66d5fd6 server_cookie=9688ee0895658de9 in_seq=0 out_seq=0 2024-03-18T15:41:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.044+0000 7f30faff5640 10 client.15936 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f30faff5640 1 -- 172.21.15.96:0/2207572743 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642eb58b2d0 con 0x5642eb6260b0 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f30faff5640 10 client.15936 handle_command_reply: tid=0 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 client.15936 shutdown 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 msgr2=0x5642eb630fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f310c009fd0 tx=0x7f310c009ac0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 shutdown_connections 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5642eb6260b0 0x5642eb69fb50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5642eb620f40 0x5642eb630fe0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5642eb620550 0x5642eb630020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 --2- 172.21.15.96:0/2207572743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642eb6198b0 0x5642eb62fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 >> 172.21.15.96:0/2207572743 conn(0x5642eb621b40 msgr2=0x5642eb61cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f30faff5640 0 client.15936 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 shutdown_connections 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 1 -- 172.21.15.96:0/2207572743 wait complete. 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.055+0000 7f3120383740 20 client.15936 trim_cache size 0 max 16384 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:41:33.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 msgr2=0x7f31000a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 0x7f31000a4e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3110090050 tx=0x7f3110009650 comp rx=0 tx=0).stop 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 msgr2=0x7f311819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3110037040 tx=0x7f3110032f40 comp rx=0 tx=0).stop 2024-03-18T15:41:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 shutdown_connections 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f31000a2940 0x7f31000a4e00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f31181022b0 0x7f311819dee0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f31181018f0 0x7f3118196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 --2- 172.21.15.96:0/1871839965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3118100700 0x7f3118196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 >> 172.21.15.96:0/1871839965 conn(0x7f31180fbc70 msgr2=0x7f3118104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 shutdown_connections 2024-03-18T15:41:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:33.056+0000 7f311d6bd640 1 -- 172.21.15.96:0/1871839965 wait complete. 2024-03-18T15:41:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:32 smithi137 ceph-mon[35868]: pgmap v395: 97 pgs: 97 active+clean; 52 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-03-18T15:41:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:33 smithi096 ceph-mon[31930]: pgmap v396: 97 pgs: 97 active+clean; 56 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-03-18T15:41:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:33 smithi096 ceph-mon[31930]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-18T15:41:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:33 smithi137 ceph-mon[35868]: pgmap v396: 97 pgs: 97 active+clean; 56 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-03-18T15:41:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:33 smithi137 ceph-mon[35868]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-18T15:41:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:33 smithi193 ceph-mon[36702]: pgmap v396: 97 pgs: 97 active+clean; 56 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 82 op/s 2024-03-18T15:41:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:33 smithi193 ceph-mon[36702]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-18T15:41:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:36 smithi137 ceph-mon[35868]: pgmap v397: 97 pgs: 97 active+clean; 56 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-18T15:41:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:36 smithi193 ceph-mon[36702]: pgmap v397: 97 pgs: 97 active+clean; 56 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-18T15:41:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:36 smithi096 ceph-mon[31930]: pgmap v397: 97 pgs: 97 active+clean; 56 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 63 op/s 2024-03-18T15:41:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:38 smithi137 ceph-mon[35868]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 817 KiB/s wr, 46 op/s 2024-03-18T15:41:38.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:38 smithi193 ceph-mon[36702]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 817 KiB/s wr, 46 op/s 2024-03-18T15:41:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:38 smithi096 ceph-mon[31930]: pgmap v398: 97 pgs: 97 active+clean; 56 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 817 KiB/s wr, 46 op/s 2024-03-18T15:41:39.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:39 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:41:39.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:39 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:41:39.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:39 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:41:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:40 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:41:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:40 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:41:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:40 smithi137 ceph-mon[35868]: pgmap v399: 97 pgs: 97 active+clean; 63 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-03-18T15:41:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:40 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:41:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:40 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:41:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:40 smithi193 ceph-mon[36702]: pgmap v399: 97 pgs: 97 active+clean; 63 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-03-18T15:41:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:40 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:41:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:40 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:41:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:40 smithi096 ceph-mon[31930]: pgmap v399: 97 pgs: 97 active+clean; 63 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-03-18T15:41:41.328 INFO:tasks.workunit.client.0.smithi096.stdout: 32 145483 447 102355 472 65131 1254 2024-03-18T15:41:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:42 smithi137 ceph-mon[35868]: pgmap v400: 97 pgs: 97 active+clean; 63 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 47 op/s 2024-03-18T15:41:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:42 smithi193 ceph-mon[36702]: pgmap v400: 97 pgs: 97 active+clean; 63 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 47 op/s 2024-03-18T15:41:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:42 smithi096 ceph-mon[31930]: pgmap v400: 97 pgs: 97 active+clean; 63 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 47 op/s 2024-03-18T15:41:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:44 smithi137 ceph-mon[35868]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-03-18T15:41:44.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:44 smithi193 ceph-mon[36702]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-03-18T15:41:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:44 smithi096 ceph-mon[31930]: pgmap v401: 97 pgs: 97 active+clean; 83 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 137 op/s 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 -- 172.21.15.96:0/3719444534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 msgr2=0x7f7b74109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/3719444534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b74109970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009ca0 tx=0x7f7b6c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 -- 172.21.15.96:0/3719444534 shutdown_connections 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/3719444534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b74109970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/3719444534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 0x7f7b74106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/3719444534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 0x7f7b741046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 -- 172.21.15.96:0/3719444534 >> 172.21.15.96:0/3719444534 conn(0x7f7b740fbcb0 msgr2=0x7f7b740fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 -- 172.21.15.96:0/3719444534 shutdown_connections 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.146+0000 7f7b7bebf640 1 -- 172.21.15.96:0/3719444534 wait complete. 2024-03-18T15:41:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 Processor -- start 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 -- start start 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 0x7f7b74197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 0x7f7b74197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410d4e0 con 0x7f7b74107520 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410d360 con 0x7f7b74104bf0 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7bebf640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7410d660 con 0x7f7b741022c0 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55182/0 (socket says 172.21.15.96:55182) 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 0x7f7b74197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 -- 172.21.15.96:0/4270087789 learned_addr learned my addr 172.21.15.96:0/4270087789 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7945a640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 0x7f7b74197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 -- 172.21.15.96:0/4270087789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 msgr2=0x7f7b74197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 0x7f7b74197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 -- 172.21.15.96:0/4270087789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 msgr2=0x7f7b74197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 0x7f7b74197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.147+0000 7f7b7a45c640 1 -- 172.21.15.96:0/4270087789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7419f680 con 0x7f7b74107520 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b7a45c640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c037040 tx=0x7f7b6c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b6affd640 1 -- 172.21.15.96:0/4270087789 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c041020 con 0x7f7b74107520 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7419f910 con 0x7f7b74107520 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b6affd640 1 -- 172.21.15.96:0/4270087789 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6c034cd0 con 0x7f7b74107520 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b6affd640 1 -- 172.21.15.96:0/4270087789 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c039040 con 0x7f7b74107520 2024-03-18T15:41:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.148+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7419fe50 con 0x7f7b74107520 2024-03-18T15:41:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 Processor -- start 2024-03-18T15:41:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b6affd640 1 -- 172.21.15.96:0/4270087789 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7b6c044020 con 0x7f7b74107520 2024-03-18T15:41:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 -- start start 2024-03-18T15:41:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b60ea0 0x563f59b63330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f59ac0a40 con 0x563f59b60ea0 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f59a69920 con 0x563f59b5bd30 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563f59abcae0 con 0x563f59b5b340 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b6affd640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 0x7f7b600a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38530/0 (socket says 172.21.15.96:38530) 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b79c5b640 1 -- 172.21.15.96:0/232403093 learned_addr learned my addr 172.21.15.96:0/232403093 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.151+0000 7f7b7945a640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b60ea0 0x563f59b63330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b7945a640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b7a45c640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 0x7f7b600a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 -- 172.21.15.96:0/232403093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 msgr2=0x563f59b5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 -- 172.21.15.96:0/232403093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b60ea0 msgr2=0x563f59b63330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b60ea0 0x563f59b63330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 -- 172.21.15.96:0/232403093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f59a81460 con 0x563f59b5bd30 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b7945a640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b6affd640 1 -- 172.21.15.96:0/4270087789 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b6c0f0930 con 0x7f7b74107520 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009d20 tx=0x7f7b70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a5915a0f407fb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b7a45c640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 0x7f7b600a4d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0b9d00 tx=0x7f7b6c090060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b5ffff640 1 -- 172.21.15.96:0/232403093 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70023070 con 0x563f59b5bd30 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b5ffff640 1 -- 172.21.15.96:0/232403093 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b70016b30 con 0x563f59b5bd30 2024-03-18T15:41:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.152+0000 7f7b5ffff640 1 -- 172.21.15.96:0/232403093 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7001eb90 con 0x563f59b5bd30 2024-03-18T15:41:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 -- 172.21.15.96:0/232403093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 msgr2=0x563f59b60830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009d20 tx=0x7f7b70011810 comp rx=0 tx=0).stop 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 -- 172.21.15.96:0/232403093 shutdown_connections 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b60ea0 0x563f59b63330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b60830 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/232403093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 -- 172.21.15.96:0/232403093 >> 172.21.15.96:0/232403093 conn(0x563f59b568b0 msgr2=0x563f59b56cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 -- 172.21.15.96:0/232403093 shutdown_connections 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.153+0000 7f7b7eb89740 1 -- 172.21.15.96:0/232403093 wait complete. 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 -- start start 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 Processor -- start 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b6cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f59b63e50 con 0x563f59b5b340 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563f59a7cd70 con 0x563f59b5bd30 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.154+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563f59b63fd0 con 0x563f59b60ea0 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7eb89740 10 client.0 _ll_register_callbacks cb 0x563f597d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7945a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38536/0 (socket says 172.21.15.96:38536) 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7945a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55194/0 (socket says 172.21.15.96:55194) 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 -- 172.21.15.96:0/3934501176 learned_addr learned my addr 172.21.15.96:0/3934501176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b6cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 -- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 msgr2=0x563f59b6cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b6cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 -- 172.21.15.96:0/3934501176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 msgr2=0x563f59b65660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 -- 172.21.15.96:0/3934501176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f597d5210 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009cf0 tx=0x7f7b7001e5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2963c59805202bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70023040 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b70016450 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.155+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b70016b30 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b70016670 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 10 client.25742 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f598a1620 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b70016b30 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f7b7002f030 con 0x563f59b5bd30 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 10 client.25742 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 20 client.25742 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59b71220 0x563f59b73670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 4 client.25742 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f59acdf80 con 0x563f59b71220 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.156+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59b71220 0x563f59b73670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.157+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59b71220 0x563f59b73670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e79e13ded7f9fe6 server_cookie=76548dbdf8faf48b in_seq=0 out_seq=0 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.157+0000 7f7b5d7fa640 10 client.25742 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/3934501176 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563f59acdf80 con 0x563f59b71220 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b5d7fa640 10 client.25742 handle_command_reply: tid=0 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 client.25742 shutdown 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 msgr2=0x563f59b65ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009cf0 tx=0x7f7b7001e5f0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 shutdown_connections 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b6cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5bd30 0x563f59b65ba0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59b71220 0x563f59b73670 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3934501176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5b340 0x563f59b65660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b5d7fa640 0 client.25742 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 >> 172.21.15.96:0/3934501176 conn(0x563f59b568b0 msgr2=0x563f59b55bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 shutdown_connections 2024-03-18T15:41:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3934501176 wait complete. 2024-03-18T15:41:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.158+0000 7f7b7eb89740 20 client.25742 trim_cache size 0 max 16384 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 Processor -- start 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 -- start start 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a6b00 con 0x563f59b5bd30 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a8760 con 0x563f59b5b340 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a71d0 con 0x563f59b60ea0 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7945a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7945a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38552/0 (socket says 172.21.15.96:38552) 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55208/0 (socket says 172.21.15.96:55208) 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7945a640 1 -- 172.21.15.96:0/3749068040 learned_addr learned my addr 172.21.15.96:0/3749068040 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.173+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 -- 172.21.15.96:0/3749068040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 msgr2=0x563f59b77f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b77f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 -- 172.21.15.96:0/3749068040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 msgr2=0x563f59b6cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 -- 172.21.15.96:0/3749068040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f59b57620 con 0x563f59b5b340 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b7945a640 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b64013010 tx=0x7f7b640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6478cdb4dc613d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b5f7fe640 1 -- 172.21.15.96:0/3749068040 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004030 con 0x563f59b5b340 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b5f7fe640 1 -- 172.21.15.96:0/3749068040 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6400fd90 con 0x563f59b5b340 2024-03-18T15:41:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.174+0000 7f7b5f7fe640 1 -- 172.21.15.96:0/3749068040 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64004c60 con 0x563f59b5b340 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3749068040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 msgr2=0x563f59b55a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b64013010 tx=0x7f7b640046c0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3749068040 shutdown_connections 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b60ea0 0x563f59b77f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/3749068040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b5b340 0x563f59b55a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3749068040 >> 172.21.15.96:0/3749068040 conn(0x563f59b568b0 msgr2=0x563f59b55320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3749068040 shutdown_connections 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- 172.21.15.96:0/3749068040 wait complete. 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 -- start start 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.175+0000 7f7b7eb89740 1 Processor -- start 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b6a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 0x563f59b6bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a7fd0 con 0x563f59b5bd30 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a7900 con 0x563f59b60ea0 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b741a4ae0 con 0x563f59b5b340 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7eb89740 10 client.0 _ll_register_callbacks cb 0x7f7b70021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55222/0 (socket says 172.21.15.96:55222) 2024-03-18T15:41:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 -- 172.21.15.96:0/489707565 learned_addr learned my addr 172.21.15.96:0/489707565 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7945a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b6a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7945a640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 0x563f59b6bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 -- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 msgr2=0x563f59b6a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b6a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 -- 172.21.15.96:0/489707565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 msgr2=0x563f59b6bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 0x563f59b6bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 -- 172.21.15.96:0/489707565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f59b6c1a0 con 0x563f59b5bd30 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b7945a640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 0x563f59b6bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b79c5b640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b70011850 tx=0x7f7b7001e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8177b2ab2d0976c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.176+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7003f060 con 0x563f59b5bd30 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b70016a10 con 0x563f59b5bd30 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6400fd90 con 0x563f59b5bd30 2024-03-18T15:41:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7b70023040 con 0x563f59b5bd30 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 10 client.15954 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f59bd7920 con 0x563f59b5bd30 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b6400fd90 con 0x563f59b5bd30 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f7b7002f040 con 0x563f59b5bd30 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 10 client.15954 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 20 client.15954 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59bd82f0 0x563f59bda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 4 client.15954 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f59ac82d0 con 0x563f59bd82f0 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.177+0000 7f7b7945a640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59bd82f0 0x563f59bda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.178+0000 7f7b5d7fa640 10 client.15954 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.178+0000 7f7b7945a640 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59bd82f0 0x563f59bda710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f0fe3121bd3347 server_cookie=baee3524ee1018a1 in_seq=0 out_seq=0 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.185+0000 7f7b5d7fa640 1 -- 172.21.15.96:0/489707565 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f59ac82d0 con 0x563f59bd82f0 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.185+0000 7f7b5d7fa640 10 client.15954 handle_command_reply: tid=0 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.185+0000 7f7b7eb89740 1 client.15954 shutdown 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 msgr2=0x563f59b6aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b70011850 tx=0x7f7b7001e500 comp rx=0 tx=0).stop 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 shutdown_connections 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f59b60ea0 0x563f59b6bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563f59bd82f0 0x563f59bda710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f59b5bd30 0x563f59b6aba0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b5d7fa640 0 client.15954 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 --2- 172.21.15.96:0/489707565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f59b5b340 0x563f59b6a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 >> 172.21.15.96:0/489707565 conn(0x563f59b568b0 msgr2=0x563f59b5a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 shutdown_connections 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 1 -- 172.21.15.96:0/489707565 wait complete. 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7eb89740 20 client.15954 trim_cache size 0 max 16384 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:41:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:41:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 msgr2=0x7f7b600a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 0x7f7b600a4d60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0b9d00 tx=0x7f7b6c090060 comp rx=0 tx=0).stop 2024-03-18T15:41:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 msgr2=0x7f7b7419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.186+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c037040 tx=0x7f7b6c032f40 comp rx=0 tx=0).stop 2024-03-18T15:41:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 shutdown_connections 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b600a28a0 0x7f7b600a4d60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74107520 0x7f7b7419f010 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b74104bf0 0x7f7b74197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 --2- 172.21.15.96:0/4270087789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b741022c0 0x7f7b74197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 >> 172.21.15.96:0/4270087789 conn(0x7f7b740fbcb0 msgr2=0x7f7b74102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 shutdown_connections 2024-03-18T15:41:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:46.187+0000 7f7b7bebf640 1 -- 172.21.15.96:0/4270087789 wait complete. 2024-03-18T15:41:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:46 smithi137 ceph-mon[35868]: pgmap v402: 97 pgs: 97 active+clean; 112 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-03-18T15:41:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:46 smithi193 ceph-mon[36702]: pgmap v402: 97 pgs: 97 active+clean; 112 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-03-18T15:41:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:46 smithi096 ceph-mon[31930]: pgmap v402: 97 pgs: 97 active+clean; 112 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-03-18T15:41:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:48 smithi137 ceph-mon[35868]: pgmap v403: 97 pgs: 97 active+clean; 129 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 6.4 MiB/s wr, 383 op/s 2024-03-18T15:41:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:48 smithi193 ceph-mon[36702]: pgmap v403: 97 pgs: 97 active+clean; 129 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 6.4 MiB/s wr, 383 op/s 2024-03-18T15:41:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:48 smithi096 ceph-mon[31930]: pgmap v403: 97 pgs: 97 active+clean; 129 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 6.4 MiB/s wr, 383 op/s 2024-03-18T15:41:50.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:41:50.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:41:50.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 -- 172.21.15.96:0/2313929380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 msgr2=0x7fc990108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 --2- 172.21.15.96:0/2313929380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc990108b20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ca0 tx=0x7fc98802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 -- 172.21.15.96:0/2313929380 shutdown_connections 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 --2- 172.21.15.96:0/2313929380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc990108b20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 --2- 172.21.15.96:0/2313929380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901018d0 0x7fc990101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 --2- 172.21.15.96:0/2313929380 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901006e0 0x7fc990100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 -- 172.21.15.96:0/2313929380 >> 172.21.15.96:0/2313929380 conn(0x7fc9900fbc70 msgr2=0x7fc9900fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 -- 172.21.15.96:0/2313929380 shutdown_connections 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.141+0000 7fc995f10640 1 -- 172.21.15.96:0/2313929380 wait complete. 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 Processor -- start 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 -- start start 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 0x7fc990197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901018d0 0x7fc9901980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010c360 con 0x7fc990102290 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010c1e0 con 0x7fc9901018d0 2024-03-18T15:41:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc995f10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc99010c4e0 con 0x7fc9901006e0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.142+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 0x7fc990197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901018d0 0x7fc9901980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55230/0 (socket says 172.21.15.96:55230) 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98ffff640 1 -- 172.21.15.96:0/2957887374 learned_addr learned my addr 172.21.15.96:0/2957887374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 -- 172.21.15.96:0/2957887374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901018d0 msgr2=0x7fc9901980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901018d0 0x7fc9901980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 -- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 msgr2=0x7fc99019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 -- 172.21.15.96:0/2957887374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99019f7a0 con 0x7fc9901006e0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98ffff640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 0x7fc990197b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9c0 tx=0x7fc98000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98d7fa640 1 -- 172.21.15.96:0/2957887374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000ccc0 con 0x7fc9901006e0 2024-03-18T15:41:50.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99019fa90 con 0x7fc9901006e0 2024-03-18T15:41:50.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98d7fa640 1 -- 172.21.15.96:0/2957887374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9800035f0 con 0x7fc9901006e0 2024-03-18T15:41:50.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc98d7fa640 1 -- 172.21.15.96:0/2957887374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98000ce60 con 0x7fc9901006e0 2024-03-18T15:41:50.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.143+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901a0030 con 0x7fc9901006e0 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.145+0000 7fc998bd8740 1 Processor -- start 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 -- start start 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98d7fa640 1 -- 172.21.15.96:0/2957887374 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc9800037a0 con 0x7fc9901006e0 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x559356577660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x55935657a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5593564d7500 con 0x559356572170 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55935647fbe0 con 0x559356577cd0 2024-03-18T15:41:50.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc998bd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5593564d35a0 con 0x559356572b60 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55246/0 (socket says 172.21.15.96:55246) 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 -- 172.21.15.96:0/3042547438 learned_addr learned my addr 172.21.15.96:0/3042547438 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x559356577660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc986ffd640 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x55935657a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 -- 172.21.15.96:0/3042547438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 msgr2=0x559356577660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x559356577660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 -- 172.21.15.96:0/3042547438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 msgr2=0x55935657a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x55935657a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 -- 172.21.15.96:0/3042547438 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593564934b0 con 0x559356572170 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98d7fa640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 0x7fc97c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc988037040 tx=0x7fc988005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f5db056a2b6b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 0x7fc97c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.146+0000 7fc98d7fa640 1 -- 172.21.15.96:0/2957887374 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc9800cef10 con 0x7fc9901006e0 2024-03-18T15:41:50.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc985ffb640 1 -- 172.21.15.96:0/3042547438 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988043070 con 0x559356572170 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc985ffb640 1 -- 172.21.15.96:0/3042547438 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988032c90 con 0x559356572170 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc985ffb640 1 -- 172.21.15.96:0/3042547438 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988039040 con 0x559356572170 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 -- 172.21.15.96:0/3042547438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 msgr2=0x559356572590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc98f7fe640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 0x7fc97c0a4cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc98009b7b0 tx=0x7fc9800d2950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc988037040 tx=0x7fc988005ef0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 -- 172.21.15.96:0/3042547438 shutdown_connections 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x55935657a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x559356577660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3042547438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356572590 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 -- 172.21.15.96:0/3042547438 >> 172.21.15.96:0/3042547438 conn(0x55935656d6e0 msgr2=0x55935656dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 -- 172.21.15.96:0/3042547438 shutdown_connections 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.147+0000 7fc998bd8740 1 -- 172.21.15.96:0/3042547438 wait complete. 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 -- start start 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 Processor -- start 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572170 0x55935657c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55935657ac20 con 0x559356572b60 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559356493160 con 0x559356577cd0 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55935657ada0 con 0x559356572170 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc998bd8740 10 client.0 _ll_register_callbacks cb 0x559356477150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55254/0 (socket says 172.21.15.96:55254) 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc986ffd640 1 -- 172.21.15.96:0/1920764421 learned_addr learned my addr 172.21.15.96:0/1920764421 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572170 0x55935657c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc986ffd640 1 -- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572170 msgr2=0x55935657c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.148+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572170 0x55935657c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc986ffd640 1 -- 172.21.15.96:0/1920764421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 msgr2=0x559356583bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc986ffd640 1 -- 172.21.15.96:0/1920764421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593562b57d0 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc978012720 tx=0x7fc978012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9346e11fd7b524c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97800f8c0 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc97800fa60 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc988032c90 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc97800fc60 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5593561eb3f0 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc988032c90 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fc978010430 con 0x559356572b60 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 20 client.15966 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565881e0 0x55935658a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 4 client.15966 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:50.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.149+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593564e4a40 con 0x5593565881e0 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.150+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565881e0 0x55935658a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.150+0000 7fc96effd640 10 client.15966 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.150+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565881e0 0x55935658a640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59438d555f075a9 server_cookie=2d30809c61d5f464 in_seq=0 out_seq=0 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc96effd640 1 -- 172.21.15.96:0/1920764421 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5593564e4a40 con 0x5593565881e0 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc96effd640 10 client.15966 handle_command_reply: tid=0 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 client.15966 shutdown 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 msgr2=0x55935657cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc978012720 tx=0x7fc978012be0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 shutdown_connections 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565881e0 0x55935658a640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572b60 0x55935657cc00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1920764421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572170 0x55935657c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 >> 172.21.15.96:0/1920764421 conn(0x55935656d6e0 msgr2=0x55935656c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc96effd640 0 client.15966 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:50.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 shutdown_connections 2024-03-18T15:41:50.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 1 -- 172.21.15.96:0/1920764421 wait complete. 2024-03-18T15:41:50.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.151+0000 7fc998bd8740 20 client.15966 trim_cache size 0 max 16384 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 Processor -- start 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 -- start start 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x55935657b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901a8ea0 con 0x559356572170 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901aab00 con 0x559356577cd0 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc998bd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901a9570 con 0x559356572b60 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55268/0 (socket says 172.21.15.96:55268) 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 -- 172.21.15.96:0/3986549162 learned_addr learned my addr 172.21.15.96:0/3986549162 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc986ffd640 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x55935657b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 -- 172.21.15.96:0/3986549162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 msgr2=0x55935657b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x55935657b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 -- 172.21.15.96:0/3986549162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 msgr2=0x559356583b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.169+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.170+0000 7fc98ffff640 1 -- 172.21.15.96:0/3986549162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593561ab0f0 con 0x559356572170 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.170+0000 7fc98ffff640 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc988002230 tx=0x7fc988005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b96b0d4160e4611 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.170+0000 7fc9857fa640 1 -- 172.21.15.96:0/3986549162 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988039040 con 0x559356572170 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.170+0000 7fc9857fa640 1 -- 172.21.15.96:0/3986549162 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98802cca0 con 0x559356572170 2024-03-18T15:41:50.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.170+0000 7fc9857fa640 1 -- 172.21.15.96:0/3986549162 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988043040 con 0x559356572170 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- 172.21.15.96:0/3986549162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 msgr2=0x55935657b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc988002230 tx=0x7fc988005ef0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- 172.21.15.96:0/3986549162 shutdown_connections 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x559356583b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x55935657b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 --2- 172.21.15.96:0/3986549162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x55935657b170 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- 172.21.15.96:0/3986549162 >> 172.21.15.96:0/3986549162 conn(0x55935656d6e0 msgr2=0x55935657c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- 172.21.15.96:0/3986549162 shutdown_connections 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- 172.21.15.96:0/3986549162 wait complete. 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.171+0000 7fc998bd8740 1 -- start start 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 Processor -- start 2024-03-18T15:41:50.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x5593565813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x5593565823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901aa370 con 0x559356572170 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901a9ca0 con 0x559356577cd0 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc9901a6e80 con 0x559356572b60 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc998bd8740 10 client.0 _ll_register_callbacks cb 0x7fc978009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x5593565813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55270/0 (socket says 172.21.15.96:55270) 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 -- 172.21.15.96:0/1395724435 learned_addr learned my addr 172.21.15.96:0/1395724435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x5593565823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 -- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 msgr2=0x5593565813c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x5593565813c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 -- 172.21.15.96:0/1395724435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 msgr2=0x5593565823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x5593565823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc98ffff640 1 -- 172.21.15.96:0/1395724435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935649e9d0 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.172+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x5593565813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc98ffff640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc988000f80 tx=0x7fc98802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a0c8dcef44ebe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988043070 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fc98805d040 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc98802cca0 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc988039040 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 10 client.15972 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559356583b70 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98802cca0 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fc988034910 con 0x559356572170 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 10 client.15972 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 20 client.15972 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565ee620 0x5593565f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 4 client.15972 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.173+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593564ded90 con 0x5593565ee620 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565ee620 0x5593565f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc96effd640 10 client.15972 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc986ffd640 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565ee620 0x5593565f0a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c81babd2047fe8 server_cookie=c742904f291980b7 in_seq=0 out_seq=0 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc96effd640 1 -- 172.21.15.96:0/1395724435 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593564ded90 con 0x5593565ee620 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc96effd640 10 client.15972 handle_command_reply: tid=0 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 client.15972 shutdown 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 msgr2=0x559356580e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc988000f80 tx=0x7fc98802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 shutdown_connections 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559356577cd0 0x5593565823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559356572b60 0x5593565813c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5593565ee620 0x5593565f0a10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 --2- 172.21.15.96:0/1395724435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559356572170 0x559356580e80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc96effd640 0 client.15972 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.174+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 >> 172.21.15.96:0/1395724435 conn(0x55935656d6e0 msgr2=0x55935657c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 shutdown_connections 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc998bd8740 1 -- 172.21.15.96:0/1395724435 wait complete. 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc998bd8740 20 client.15972 trim_cache size 0 max 16384 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:41:50.183 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:50.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 msgr2=0x7fc97c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 0x7fc97c0a4cc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc98009b7b0 tx=0x7fc9800d2950 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 msgr2=0x7fc990197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 0x7fc990197b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc98000e9c0 tx=0x7fc98000ee80 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 shutdown_connections 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc97c0a2800 0x7fc97c0a4cc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990102290 0x7fc99019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9901018d0 0x7fc9901980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 --2- 172.21.15.96:0/2957887374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9901006e0 0x7fc990197b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 >> 172.21.15.96:0/2957887374 conn(0x7fc9900fbc70 msgr2=0x7fc9900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 shutdown_connections 2024-03-18T15:41:50.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.175+0000 7fc995f10640 1 -- 172.21.15.96:0/2957887374 wait complete. 2024-03-18T15:41:50.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:41:50.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:41:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 -- 172.21.15.96:0/964521672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 msgr2=0x7f2de0100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 --2- 172.21.15.96:0/964521672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0100280 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009900 tx=0x7f2dd002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 -- 172.21.15.96:0/964521672 shutdown_connections 2024-03-18T15:41:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 --2- 172.21.15.96:0/964521672 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101a00 0x7f2de0108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 --2- 172.21.15.96:0/964521672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101040 0x7f2de01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 --2- 172.21.15.96:0/964521672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0100280 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 -- 172.21.15.96:0/964521672 >> 172.21.15.96:0/964521672 conn(0x7f2de00fb420 msgr2=0x7f2de00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 -- 172.21.15.96:0/964521672 shutdown_connections 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.290+0000 7f2de67cd640 1 -- 172.21.15.96:0/964521672 wait complete. 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 Processor -- start 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 -- start start 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101040 0x7f2de0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 0x7f2de019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c3e0 con 0x7f2de00ffe60 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c260 con 0x7f2de0101a00 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de67cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2de010c560 con 0x7f2de0101040 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2ddffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2ddffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55286/0 (socket says 172.21.15.96:55286) 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2ddffff640 1 -- 172.21.15.96:0/433835135 learned_addr learned my addr 172.21.15.96:0/433835135 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2de4d6a640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 0x7f2de019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.291+0000 7f2ddf7fe640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101040 0x7f2de0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 -- 172.21.15.96:0/433835135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101040 msgr2=0x7f2de0196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101040 0x7f2de0196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 -- 172.21.15.96:0/433835135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 msgr2=0x7f2de019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 0x7f2de019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 -- 172.21.15.96:0/433835135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de019e650 con 0x7f2de00ffe60 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2de4d6a640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 0x7f2de019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddffff640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0000c00 tx=0x7f2dd002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0041070 con 0x7f2de00ffe60 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd0004450 con 0x7f2de00ffe60 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de0108ce0 con 0x7f2de00ffe60 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0034440 con 0x7f2de00ffe60 2024-03-18T15:41:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.292+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de0109220 con 0x7f2de00ffe60 2024-03-18T15:41:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.293+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de0068a20 con 0x7f2de00ffe60 2024-03-18T15:41:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.294+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2dd0044070 con 0x7f2de00ffe60 2024-03-18T15:41:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.295+0000 7f2ddd7fa640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 0x7f2dc80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.295+0000 7f2ddf7fe640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 0x7f2dc80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.295+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2dd00f04d0 con 0x7f2de00ffe60 2024-03-18T15:41:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.296+0000 7f2ddf7fe640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 0x7f2dc80a4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2de0197f40 tx=0x7f2dd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.300+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f2dd00bcf80 con 0x7f2de00ffe60 2024-03-18T15:41:50.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.536+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2de019e8e0 con 0x7f2de00ffe60 2024-03-18T15:41:50.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.537+0000 7f2ddd7fa640 1 -- 172.21.15.96:0/433835135 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11945 (secure 0 0 0) 0x7f2dd00c1020 con 0x7f2de00ffe60 2024-03-18T15:41:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:41:50.580 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:41:28.657174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:41:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 29 2024-03-18T15:41:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 msgr2=0x7f2dc80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 0x7f2dc80a4d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2de0197f40 tx=0x7f2dd4005f90 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 msgr2=0x7f2de0196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0000c00 tx=0x7f2dd002c640 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 shutdown_connections 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2dc80a2880 0x7f2dc80a4d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2de0101a00 0x7f2de019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de0101040 0x7f2de0196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 --2- 172.21.15.96:0/433835135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2de00ffe60 0x7f2de0196a30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 >> 172.21.15.96:0/433835135 conn(0x7f2de00fb420 msgr2=0x7f2de01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 shutdown_connections 2024-03-18T15:41:50.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.539+0000 7f2de67cd640 1 -- 172.21.15.96:0/433835135 wait complete. 2024-03-18T15:41:50.584 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 -- 172.21.15.96:0/1400964646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b00ff730 msgr2=0x7f35b00ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/1400964646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b00ff730 0x7f35b00ffbb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 -- 172.21.15.96:0/1400964646 shutdown_connections 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/1400964646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b01005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/1400964646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b00ff730 0x7f35b00ffbb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/1400964646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b0101c40 0x7f35b0102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 -- 172.21.15.96:0/1400964646 >> 172.21.15.96:0/1400964646 conn(0x7f35b00fb2d0 msgr2=0x7f35b00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 -- 172.21.15.96:0/1400964646 shutdown_connections 2024-03-18T15:41:50.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.706+0000 7f35b5f9f640 1 -- 172.21.15.96:0/1400964646 wait complete. 2024-03-18T15:41:50.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.707+0000 7f35b5f9f640 1 Processor -- start 2024-03-18T15:41:50.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.707+0000 7f35b5f9f640 1 -- start start 2024-03-18T15:41:50.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.707+0000 7f35b5f9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.707+0000 7f35b5f9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b0105720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.707+0000 7f35b5f9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b0101c40 0x7f35b010c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b5f9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010f550 con 0x7f35b0101c40 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b5f9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010f3d0 con 0x7f35b00ff730 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b5f9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35b010f6d0 con 0x7f35b01000f0 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38664/0 (socket says 172.21.15.96:38664) 2024-03-18T15:41:50.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 -- 172.21.15.96:0/410274040 learned_addr learned my addr 172.21.15.96:0/410274040 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b579e640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b0101c40 0x7f35b010c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35affff640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b0105720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 -- 172.21.15.96:0/410274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 msgr2=0x7f35b0105720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b0105720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 -- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b0101c40 msgr2=0x7f35b010c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b0101c40 0x7f35b010c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 -- 172.21.15.96:0/410274040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01ae450 con 0x7f35b00ff730 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35affff640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b0105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f35a400ea90 tx=0x7f35a400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35adffb640 1 -- 172.21.15.96:0/410274040 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a400ce40 con 0x7f35b00ff730 2024-03-18T15:41:50.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.708+0000 7f35adffb640 1 -- 172.21.15.96:0/410274040 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a40035f0 con 0x7f35b00ff730 2024-03-18T15:41:50.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.709+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01ae740 con 0x7f35b00ff730 2024-03-18T15:41:50.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.709+0000 7f35adffb640 1 -- 172.21.15.96:0/410274040 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4010690 con 0x7f35b00ff730 2024-03-18T15:41:50.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.709+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01aecb0 con 0x7f35b00ff730 2024-03-18T15:41:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.712+0000 7f35adffb640 1 -- 172.21.15.96:0/410274040 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f35a4010830 con 0x7f35b00ff730 2024-03-18T15:41:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35adffb640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 0x7f35a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 Processor -- start 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 -- start start 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35affff640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 0x7f35a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b804e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b809f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55711b769490 con 0x55711b805430 2024-03-18T15:41:50.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35adffb640 1 -- 172.21.15.96:0/410274040 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f35a40cee40 con 0x7f35b00ff730 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55711b711c90 con 0x55711b804a40 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b9de0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55711b765530 con 0x55711b80a5a0 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:46706/0 (socket says 172.21.15.96:46706) 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b579e640 1 -- 172.21.15.96:0/1181896356 learned_addr learned my addr 172.21.15.96:0/1181896356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35affff640 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b809f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.713+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b804e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 -- 172.21.15.96:0/1181896356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 msgr2=0x55711b804e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b804e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 -- 172.21.15.96:0/1181896356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 msgr2=0x55711b809f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b809f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 -- 172.21.15.96:0/1181896356 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711b7cc500 con 0x55711b80a5a0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35b579e640 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc08600de3c0f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f35affff640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 0x7f35a00a4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f359c02c7b0 tx=0x7f359c02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f359affd640 1 -- 172.21.15.96:0/1181896356 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8004d20 con 0x55711b80a5a0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f359affd640 1 -- 172.21.15.96:0/1181896356 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a8004050 con 0x55711b80a5a0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.714+0000 7f359affd640 1 -- 172.21.15.96:0/1181896356 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8005280 con 0x55711b80a5a0 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 -- 172.21.15.96:0/1181896356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 msgr2=0x55711b80ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009fd0 tx=0x7f35a80118d0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 -- 172.21.15.96:0/1181896356 shutdown_connections 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b80ca30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b809f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1181896356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b804e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 -- 172.21.15.96:0/1181896356 >> 172.21.15.96:0/1181896356 conn(0x55711b7fffb0 msgr2=0x55711b8003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 -- 172.21.15.96:0/1181896356 shutdown_connections 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.715+0000 7f35b9de0740 1 -- 172.21.15.96:0/1181896356 wait complete. 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.716+0000 7f35b9de0740 1 -- start start 2024-03-18T15:41:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.716+0000 7f35b9de0740 1 Processor -- start 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.716+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b81fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.716+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b827460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b9de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55711b81e560 con 0x55711b805430 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b9de0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55711b725230 con 0x55711b804a40 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b9de0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55711b81e6e0 con 0x55711b80a5a0 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b9de0740 10 client.0 _ll_register_callbacks cb 0x55711b761010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b827460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55324/0 (socket says 172.21.15.96:55324) 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 -- 172.21.15.96:0/688187929 learned_addr learned my addr 172.21.15.96:0/688187929 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b81fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 -- 172.21.15.96:0/688187929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 msgr2=0x55711b827460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b827460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 -- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 msgr2=0x55711b81fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b81fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 -- 172.21.15.96:0/688187929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711b734100 con 0x55711b805430 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b81fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35b579e640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b827460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f35affff640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f359c005ec0 tx=0x7f359c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12c120023f7f3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f358ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.717+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c053060 con 0x55711b805430 2024-03-18T15:41:50.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a8004050 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f359c004990 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f359c004b90 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 10 client.15990 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55711b47d3f0 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a8004050 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f359c03f470 con 0x55711b805430 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 10 client.15990 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 10 client.15990 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 20 client.15990 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b82bd50 0x55711b82e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 4 client.15990 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55711b7769d0 con 0x55711b82bd50 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.718+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b82bd50 0x55711b82e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.719+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b82bd50 0x55711b82e1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cda53952fe21960 server_cookie=a4667e2df834b91f in_seq=0 out_seq=0 2024-03-18T15:41:50.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.719+0000 7f358ffff640 10 client.15990 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.719+0000 7f358ffff640 1 -- 172.21.15.96:0/688187929 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55711b7769d0 con 0x55711b82bd50 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.719+0000 7f358ffff640 10 client.15990 handle_command_reply: tid=0 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 client.15990 shutdown 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 msgr2=0x55711b820420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f359c005ec0 tx=0x7f359c005fd0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 shutdown_connections 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b82bd50 0x55711b82e1d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b827460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b805430 0x55711b820420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f358ffff640 0 client.15990 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 --2- 172.21.15.96:0/688187929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b804a40 0x55711b81fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 >> 172.21.15.96:0/688187929 conn(0x55711b7fffb0 msgr2=0x55711b7ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 shutdown_connections 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 1 -- 172.21.15.96:0/688187929 wait complete. 2024-03-18T15:41:50.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.720+0000 7f35b9de0740 20 client.15990 trim_cache size 0 max 16384 2024-03-18T15:41:50.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 Processor -- start 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 -- start start 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 0x55711b827420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b7fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01af120 con 0x55711b804a40 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01b0d80 con 0x55711b805430 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.739+0000 7f35b9de0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01af7f0 con 0x55711b80a5a0 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55332/0 (socket says 172.21.15.96:55332) 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 -- 172.21.15.96:0/2672120655 learned_addr learned my addr 172.21.15.96:0/2672120655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b579e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b7fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35affff640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 0x55711b827420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 -- 172.21.15.96:0/2672120655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 msgr2=0x55711b7fedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b7fedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 -- 172.21.15.96:0/2672120655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 msgr2=0x55711b827420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 0x55711b827420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 -- 172.21.15.96:0/2672120655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711b5477d0 con 0x55711b804a40 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b579e640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b7fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35affff640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 0x55711b827420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35a40ccc80 tx=0x7f35a40045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b530271183ec069a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f359a7fc640 1 -- 172.21.15.96:0/2672120655 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40a3f50 con 0x55711b804a40 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f359a7fc640 1 -- 172.21.15.96:0/2672120655 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4099850 con 0x55711b804a40 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.740+0000 7f359a7fc640 1 -- 172.21.15.96:0/2672120655 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4086200 con 0x55711b804a40 2024-03-18T15:41:50.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 -- 172.21.15.96:0/2672120655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 msgr2=0x55711b7fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f35a40ccc80 tx=0x7f35a40045d0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 -- 172.21.15.96:0/2672120655 shutdown_connections 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b80a5a0 0x55711b7fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b805430 0x55711b827420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 --2- 172.21.15.96:0/2672120655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b7fe8b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 -- 172.21.15.96:0/2672120655 >> 172.21.15.96:0/2672120655 conn(0x55711b7fffb0 msgr2=0x55711b806890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 -- 172.21.15.96:0/2672120655 shutdown_connections 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.741+0000 7f35b9de0740 1 -- 172.21.15.96:0/2672120655 wait complete. 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 -- start start 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 Processor -- start 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b8244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b80a5a0 0x55711b8900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01b05f0 con 0x55711b804a40 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01aff20 con 0x55711b80a5a0 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35b4f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b8244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:46720/0 (socket says 172.21.15.96:46720) 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 -- 172.21.15.96:0/1952545840 learned_addr learned my addr 172.21.15.96:0/1952545840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.742+0000 7f35b9de0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0071bc0 con 0x55711b805430 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35b579e640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b80a5a0 0x55711b8900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35b9de0740 10 client.0 _ll_register_callbacks cb 0x7f35a80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 -- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b80a5a0 msgr2=0x55711b8900c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b80a5a0 0x55711b8900c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 -- 172.21.15.96:0/1952545840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 msgr2=0x55711b8244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b8244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 -- 172.21.15.96:0/1952545840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711b890630 con 0x55711b805430 2024-03-18T15:41:50.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f35affff640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f359c03f930 tx=0x7f359c03fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa0341c36f9bdf99 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f358ffff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.743+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c053040 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f359c004990 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a4099850 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f359c03f680 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 10 client.25840 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55711b891910 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4099850 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f359c043450 con 0x55711b805430 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 10 client.25840 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 10 client.25840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 20 client.25840 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b892340 0x55711b894760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 4 client.25840 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55711b770d20 con 0x55711b892340 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.744+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b892340 0x55711b894760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f358ffff640 10 client.25840 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b4f9d640 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b892340 0x55711b894760 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b14144a5a30bcef server_cookie=e9a1578a1872adac in_seq=0 out_seq=0 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f358ffff640 1 -- 172.21.15.96:0/1952545840 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55711b770d20 con 0x55711b892340 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f358ffff640 10 client.25840 handle_command_reply: tid=0 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 client.25840 shutdown 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 msgr2=0x55711b824a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f359c03f930 tx=0x7f359c03fdd0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 shutdown_connections 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55711b892340 0x55711b894760 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55711b80a5a0 0x55711b8900c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55711b805430 0x55711b824a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 --2- 172.21.15.96:0/1952545840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55711b804a40 0x55711b8244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f358ffff640 0 client.25840 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 >> 172.21.15.96:0/1952545840 conn(0x55711b7fffb0 msgr2=0x55711b7fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 shutdown_connections 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 1 -- 172.21.15.96:0/1952545840 wait complete. 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.745+0000 7f35b9de0740 20 client.25840 trim_cache size 0 max 16384 2024-03-18T15:41:50.753 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 msgr2=0x7f35a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 0x7f35a00a4d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f359c02c7b0 tx=0x7f359c02c880 comp rx=0 tx=0).stop 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 msgr2=0x7f35b01051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f35a400ea90 tx=0x7f35a400ef50 comp rx=0 tx=0).stop 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 shutdown_connections 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35a00a2880 0x7f35a00a4d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35b0101c40 0x7f35b010c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b01000f0 0x7f35b0105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 --2- 172.21.15.96:0/410274040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35b00ff730 0x7f35b01051e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 >> 172.21.15.96:0/410274040 conn(0x7f35b00fb2d0 msgr2=0x7f35b00fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 shutdown_connections 2024-03-18T15:41:50.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.746+0000 7f35b5f9f640 1 -- 172.21.15.96:0/410274040 wait complete. 2024-03-18T15:41:50.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:41:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:50 smithi137 ceph-mon[35868]: pgmap v404: 97 pgs: 97 active+clean; 150 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 494 op/s 2024-03-18T15:41:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:50 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/433835135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 -- 172.21.15.96:0/2631560551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04103a20 msgr2=0x7f4d04103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 --2- 172.21.15.96:0/2631560551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04103a20 0x7f4d04103e40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009900 tx=0x7f4cf402bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 -- 172.21.15.96:0/2631560551 shutdown_connections 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 --2- 172.21.15.96:0/2631560551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d041055d0 0x7f4d0410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 --2- 172.21.15.96:0/2631560551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d04104c10 0x7f4d04105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 --2- 172.21.15.96:0/2631560551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04103a20 0x7f4d04103e40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 -- 172.21.15.96:0/2631560551 >> 172.21.15.96:0/2631560551 conn(0x7f4d040fef90 msgr2=0x7f4d041013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 -- 172.21.15.96:0/2631560551 shutdown_connections 2024-03-18T15:41:50.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 -- 172.21.15.96:0/2631560551 wait complete. 2024-03-18T15:41:50.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.875+0000 7f4d09263640 1 Processor -- start 2024-03-18T15:41:50.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 -- start start 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d04103a20 0x7f4d04199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d041055d0 0x7f4d041a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410f6a0 con 0x7f4d04104c10 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410f520 con 0x7f4d041055d0 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d09263640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0410f820 con 0x7f4d04103a20 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55356/0 (socket says 172.21.15.96:55356) 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 -- 172.21.15.96:0/1508469028 learned_addr learned my addr 172.21.15.96:0/1508469028 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d04103a20 0x7f4d04199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d037fe640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d041055d0 0x7f4d041a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 -- 172.21.15.96:0/1508469028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d04103a20 msgr2=0x7f4d04199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d04103a20 0x7f4d04199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 -- 172.21.15.96:0/1508469028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d041055d0 msgr2=0x7f4d041a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d041055d0 0x7f4d041a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 -- 172.21.15.96:0/1508469028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d041a1960 con 0x7f4d04104c10 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.876+0000 7f4d027fc640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ea60 tx=0x7f4cf800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.877+0000 7f4cebfff640 1 -- 172.21.15.96:0/1508469028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf800cde0 con 0x7f4d04104c10 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.877+0000 7f4cebfff640 1 -- 172.21.15.96:0/1508469028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf80035f0 con 0x7f4d04104c10 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.877+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d041a1c50 con 0x7f4d04104c10 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.877+0000 7f4cebfff640 1 -- 172.21.15.96:0/1508469028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8010670 con 0x7f4d04104c10 2024-03-18T15:41:50.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.877+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d041a21a0 con 0x7f4d04104c10 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.879+0000 7f4cebfff640 1 -- 172.21.15.96:0/1508469028 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4cf80037a0 con 0x7f4d04104c10 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4cebfff640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 0x7f4cec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 Processor -- start 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 -- start start 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 0x7f4cec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4cebfff640 1 -- 172.21.15.96:0/1508469028 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4cf80ced70 con 0x7f4d04104c10 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be6cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55701bdd1490 con 0x55701be723b0 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55701bd79c90 con 0x55701be6c850 2024-03-18T15:41:50.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d0be48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55701bdcd530 con 0x55701be6d240 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be6cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55360/0 (socket says 172.21.15.96:55360) 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d037fe640 1 -- 172.21.15.96:0/3643446210 learned_addr learned my addr 172.21.15.96:0/3643446210 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d027fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.880+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 0x7f4cec0a4cc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 -- 172.21.15.96:0/3643446210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 msgr2=0x55701be71d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be71d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 -- 172.21.15.96:0/3643446210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 msgr2=0x55701be6cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be6cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 -- 172.21.15.96:0/3643446210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701be34500 con 0x55701be723b0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be6cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4d037fe640 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009fd0 tx=0x7f4cfc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543afb01877fafee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4ce97fa640 1 -- 172.21.15.96:0/3643446210 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc004d20 con 0x55701be723b0 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4ce97fa640 1 -- 172.21.15.96:0/3643446210 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cfc004050 con 0x55701be723b0 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.881+0000 7f4ce97fa640 1 -- 172.21.15.96:0/3643446210 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc005280 con 0x55701be723b0 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- 172.21.15.96:0/3643446210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 msgr2=0x55701be74840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009fd0 tx=0x7f4cfc0118d0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- 172.21.15.96:0/3643446210 shutdown_connections 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be74840 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be71d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3643446210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be6cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- 172.21.15.96:0/3643446210 >> 172.21.15.96:0/3643446210 conn(0x55701be67dc0 msgr2=0x55701be681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- 172.21.15.96:0/3643446210 shutdown_connections 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- 172.21.15.96:0/3643446210 wait complete. 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 -- start start 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.882+0000 7f4d0be48740 1 Processor -- start 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be77290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55701be75360 con 0x55701be723b0 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55701bd8d230 con 0x55701be6c850 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55701be754e0 con 0x55701be6d240 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 10 client.0 _ll_register_callbacks cb 0x55701bdc9010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55372/0 (socket says 172.21.15.96:55372) 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 -- 172.21.15.96:0/2352920848 learned_addr learned my addr 172.21.15.96:0/2352920848 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d027fc640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be77290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 -- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 msgr2=0x55701be77290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be77290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 -- 172.21.15.96:0/2352920848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 msgr2=0x55701be76d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be76d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 -- 172.21.15.96:0/2352920848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701bd9c100 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d037fe640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc003c60 tx=0x7f4cfc003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5b7f5d98a1528f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4ce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc029020 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4cfc004610 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cfc004050 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.883+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cfc034050 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 10 client.16008 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55701bae53f0 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cfc004050 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f4cfc025590 con 0x55701be723b0 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 10 client.16008 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 10 client.16008 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 20 client.16008 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701be84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 4 client.16008 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:50.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55701bdde9d0 con 0x55701be829b0 2024-03-18T15:41:50.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.884+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701be84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.885+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701be84e20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0e868f5f0e7b04c server_cookie=f078951037260f6c in_seq=0 out_seq=0 2024-03-18T15:41:50.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.885+0000 7f4ce6ff5640 10 client.16008 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.885+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/2352920848 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55701bdde9d0 con 0x55701be829b0 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.885+0000 7f4ce6ff5640 10 client.16008 handle_command_reply: tid=0 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 client.16008 shutdown 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 msgr2=0x55701be7e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc003c60 tx=0x7f4cfc003c90 comp rx=0 tx=0).stop 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 shutdown_connections 2024-03-18T15:41:50.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701be84e20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be723b0 0x55701be7e290 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4ce6ff5640 0 client.16008 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be77290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2352920848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be6c850 0x55701be76d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 >> 172.21.15.96:0/2352920848 conn(0x55701be67dc0 msgr2=0x55701be66fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 shutdown_connections 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 1 -- 172.21.15.96:0/2352920848 wait complete. 2024-03-18T15:41:50.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.886+0000 7f4d0be48740 20 client.16008 trim_cache size 0 max 16384 2024-03-18T15:41:50.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 Processor -- start 2024-03-18T15:41:50.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 -- start start 2024-03-18T15:41:50.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701be6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041ab040 con 0x55701be6c850 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041acca0 con 0x55701be723b0 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d0be48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041ab710 con 0x55701be6d240 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.904+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701be6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d027fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55384/0 (socket says 172.21.15.96:55384) 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 -- 172.21.15.96:0/2497982082 learned_addr learned my addr 172.21.15.96:0/2497982082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 -- 172.21.15.96:0/2497982082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 msgr2=0x55701be7e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 -- 172.21.15.96:0/2497982082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 msgr2=0x55701be6ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701be6ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 -- 172.21.15.96:0/2497982082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701be768f0 con 0x55701be6c850 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d037fe640 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701be6ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4cf402c7b0 tx=0x7f4cf4043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a18a45be73c5c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4ce8ff9640 1 -- 172.21.15.96:0/2497982082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4054d90 con 0x55701be6c850 2024-03-18T15:41:50.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4ce8ff9640 1 -- 172.21.15.96:0/2497982082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf4005040 con 0x55701be6c850 2024-03-18T15:41:50.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.905+0000 7f4ce8ff9640 1 -- 172.21.15.96:0/2497982082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4004a90 con 0x55701be6c850 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- 172.21.15.96:0/2497982082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 msgr2=0x55701be6e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4cf402c7b0 tx=0x7f4cf4043000 comp rx=0 tx=0).stop 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- 172.21.15.96:0/2497982082 shutdown_connections 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701be6ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 --2- 172.21.15.96:0/2497982082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be6e740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- 172.21.15.96:0/2497982082 >> 172.21.15.96:0/2497982082 conn(0x55701be67dc0 msgr2=0x55701be681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- 172.21.15.96:0/2497982082 shutdown_connections 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- 172.21.15.96:0/2497982082 wait complete. 2024-03-18T15:41:50.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.906+0000 7f4d0be48740 1 -- start start 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 Processor -- start 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701beef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041ac510 con 0x55701be6c850 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041abe40 con 0x55701be723b0 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d041a9020 con 0x55701be6d240 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d0be48740 10 client.0 _ll_register_callbacks cb 0x7f4cf4041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55388/0 (socket says 172.21.15.96:55388) 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 -- 172.21.15.96:0/3950867503 learned_addr learned my addr 172.21.15.96:0/3950867503 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d027fc640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d037fe640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701beef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 -- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 msgr2=0x55701be7b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 -- 172.21.15.96:0/3950867503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 msgr2=0x55701beef520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701beef520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 -- 172.21.15.96:0/3950867503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55701bbaf7d0 con 0x55701be6c850 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d037fe640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701beef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4d02ffd640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4036040 tx=0x7f4cf402ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2bd09a1c6cdcda server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.907+0000 7f4ce6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4060070 con 0x55701be6c850 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4cf4054620 con 0x55701be6c850 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf4005040 con 0x55701be6c850 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cf406b050 con 0x55701be6c850 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 10 client.16014 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:50.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55701bef0860 con 0x55701be6c850 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cf4005040 con 0x55701be6c850 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f4cf4046040 con 0x55701be6c850 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 10 client.16014 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 20 client.16014 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701bef3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 4 client.16014 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55701bdd8d20 con 0x55701be829b0 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.908+0000 7f4d027fc640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701bef3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4ce6ff5640 10 client.16014 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4d027fc640 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701bef3220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abeb4a2498f6d996 server_cookie=3272d22856bc3041 in_seq=0 out_seq=0 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4ce6ff5640 1 -- 172.21.15.96:0/3950867503 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55701bdd8d20 con 0x55701be829b0 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4ce6ff5640 10 client.16014 handle_command_reply: tid=0 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4d0be48740 1 client.16014 shutdown 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 msgr2=0x55701be7ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.909+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4036040 tx=0x7f4cf402ce00 comp rx=0 tx=0).stop 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 shutdown_connections 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55701be829b0 0x55701bef3220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55701be723b0 0x55701beef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55701be6d240 0x55701be7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4ce6ff5640 0 client.16014 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 --2- 172.21.15.96:0/3950867503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55701be6c850 0x55701be7ae50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 >> 172.21.15.96:0/3950867503 conn(0x55701be67dc0 msgr2=0x55701be6e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 shutdown_connections 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 1 -- 172.21.15.96:0/3950867503 wait complete. 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.910+0000 7f4d0be48740 20 client.16014 trim_cache size 0 max 16384 2024-03-18T15:41:50.916 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 msgr2=0x7f4cec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 0x7f4cec0a4cc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4000c00 tx=0x7f4cf4005fd0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 msgr2=0x7f4d0419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:50.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800ea60 tx=0x7f4cf800ef20 comp rx=0 tx=0).stop 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 shutdown_connections 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4cec0a2800 0x7f4cec0a4cc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d041055d0 0x7f4d041a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d04104c10 0x7f4d0419a280 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 --2- 172.21.15.96:0/1508469028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d04103a20 0x7f4d04199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 >> 172.21.15.96:0/1508469028 conn(0x7f4d040fef90 msgr2=0x7f4d04107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 shutdown_connections 2024-03-18T15:41:50.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:50.911+0000 7f4d09263640 1 -- 172.21.15.96:0/1508469028 wait complete. 2024-03-18T15:41:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:50 smithi193 ceph-mon[36702]: pgmap v404: 97 pgs: 97 active+clean; 150 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 494 op/s 2024-03-18T15:41:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:50 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/433835135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:50.924 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:50 smithi096 ceph-mon[31930]: pgmap v404: 97 pgs: 97 active+clean; 150 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 494 op/s 2024-03-18T15:41:50.924 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:50 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/433835135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:41:50.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:41:51.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.035+0000 7f4be5f76640 1 -- 172.21.15.96:0/245739354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0100720 msgr2=0x7f4be0100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.035+0000 7f4be5f76640 1 --2- 172.21.15.96:0/245739354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0100720 0x7f4be0100b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009900 tx=0x7f4bd002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 -- 172.21.15.96:0/245739354 shutdown_connections 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 --2- 172.21.15.96:0/245739354 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be01022d0 0x7f4be0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 --2- 172.21.15.96:0/245739354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be0101910 0x7f4be0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 --2- 172.21.15.96:0/245739354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0100720 0x7f4be0100b40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 -- 172.21.15.96:0/245739354 >> 172.21.15.96:0/245739354 conn(0x7f4be00fbc90 msgr2=0x7f4be00fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 -- 172.21.15.96:0/245739354 shutdown_connections 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 -- 172.21.15.96:0/245739354 wait complete. 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 Processor -- start 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.036+0000 7f4be5f76640 1 -- start start 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be0100720 0x7f4be0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 0x7f4be019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4be010c3a0 con 0x7f4be01022d0 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4be010c220 con 0x7f4be0101910 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4be5f76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4be010c520 con 0x7f4be0100720 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 0x7f4be019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38796/0 (socket says 172.21.15.96:38796) 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 -- 172.21.15.96:0/2002099304 learned_addr learned my addr 172.21.15.96:0/2002099304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be0100720 0x7f4be0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 -- 172.21.15.96:0/2002099304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be0100720 msgr2=0x7f4be0197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be0100720 0x7f4be0197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 -- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 msgr2=0x7f4be019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 0x7f4be019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 -- 172.21.15.96:0/2002099304 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be019f810 con 0x7f4be0101910 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdffff640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 0x7f4be019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.037+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400e9f0 tx=0x7f4bd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.038+0000 7f4bdcff9640 1 -- 172.21.15.96:0/2002099304 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400cd80 con 0x7f4be0101910 2024-03-18T15:41:51.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.038+0000 7f4bdcff9640 1 -- 172.21.15.96:0/2002099304 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd4010910 con 0x7f4be0101910 2024-03-18T15:41:51.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.038+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be019fb00 con 0x7f4be0101910 2024-03-18T15:41:51.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.038+0000 7f4bdcff9640 1 -- 172.21.15.96:0/2002099304 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4018990 con 0x7f4be0101910 2024-03-18T15:41:51.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.038+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be01a00a0 con 0x7f4be0101910 2024-03-18T15:41:51.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 Processor -- start 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 -- start start 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 0x56451fa97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 0x56451fa9a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56451f9f7490 con 0x56451fa92850 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56451f99fc90 con 0x56451fa983b0 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4be8c34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56451f9f3530 con 0x56451fa93240 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.040+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55410/0 (socket says 172.21.15.96:55410) 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 -- 172.21.15.96:0/112807995 learned_addr learned my addr 172.21.15.96:0/112807995 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 0x56451fa97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdcff9640 1 -- 172.21.15.96:0/2002099304 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4bd4018b30 con 0x7f4be0101910 2024-03-18T15:41:51.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 -- 172.21.15.96:0/112807995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 msgr2=0x56451fa97d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 0x56451fa97d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 0x56451fa9a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 -- 172.21.15.96:0/112807995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 msgr2=0x56451fa9a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 0x56451fa9a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 -- 172.21.15.96:0/112807995 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451fa5a500 con 0x56451fa92850 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 0x56451fa97d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 0x56451fa9a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdffff640 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80117e0 tx=0x7f4bd8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ad8fcd7ec578a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bdcff9640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 0x7f4bc80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bc5ffb640 1 -- 172.21.15.96:0/112807995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8011310 con 0x56451fa92850 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.041+0000 7f4bc5ffb640 1 -- 172.21.15.96:0/112807995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd8011310 con 0x56451fa92850 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4bc5ffb640 1 -- 172.21.15.96:0/112807995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8004250 con 0x56451fa92850 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 0x7f4bc80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4bdcff9640 1 -- 172.21.15.96:0/2002099304 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4bd40ced20 con 0x7f4be0101910 2024-03-18T15:41:51.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 -- 172.21.15.96:0/112807995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 msgr2=0x56451fa92c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 0x7f4bc80a4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005ab0 tx=0x7f4bd002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80117e0 tx=0x7f4bd8011ca0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 -- 172.21.15.96:0/112807995 shutdown_connections 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa983b0 0x56451fa9a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa93240 0x56451fa97d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 --2- 172.21.15.96:0/112807995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa92850 0x56451fa92c70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.042+0000 7f4be8c34740 1 -- 172.21.15.96:0/112807995 >> 172.21.15.96:0/112807995 conn(0x56451fa8ddc0 msgr2=0x56451fa8e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- 172.21.15.96:0/112807995 shutdown_connections 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- 172.21.15.96:0/112807995 wait complete. 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- start start 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 Processor -- start 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faa5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451faacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56451fa9b360 con 0x56451fa983b0 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56451f9b3230 con 0x56451fa93240 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.043+0000 7f4be8c34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56451fa9b4e0 con 0x56451fa92850 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4be8c34740 10 client.0 _ll_register_callbacks cb 0x56451f9ef010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:46796/0 (socket says 172.21.15.96:46796) 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faa5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdf7fe640 1 -- 172.21.15.96:0/4025498403 learned_addr learned my addr 172.21.15.96:0/4025498403 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdffff640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451faacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 -- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 msgr2=0x56451faa5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 -- 172.21.15.96:0/4025498403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 msgr2=0x56451faacb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451faacb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 -- 172.21.15.96:0/4025498403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451f9c2100 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faa5ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cc840 tx=0x7f4bd409b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbe9667ec110cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd409f7e0 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4bd4099a40 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.044+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd8011310 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bd40a3e30 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56451f70b3f0 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd8011310 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f4bd4097490 con 0x56451fa93240 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:41:51.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 20 client.25817 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fab1220 0x56451fab36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 4 client.25817 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56451fa049d0 con 0x56451fab1220 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.045+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fab1220 0x56451fab36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.046+0000 7f4bbeffd640 10 client.25817 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:51.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.046+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fab1220 0x56451fab36a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a3e97d8fdce4913 server_cookie=538e1c0edf7125a4 in_seq=0 out_seq=0 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.049+0000 7f4bbeffd640 1 -- 172.21.15.96:0/4025498403 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56451fa049d0 con 0x56451fab1220 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.049+0000 7f4bbeffd640 10 client.25817 handle_command_reply: tid=0 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.049+0000 7f4be8c34740 1 client.25817 shutdown 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.049+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 msgr2=0x56451faa5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.049+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faa5ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40cc840 tx=0x7f4bd409b340 comp rx=0 tx=0).stop 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 shutdown_connections 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fab1220 0x56451fab36a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451faacb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faa5ac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4bbeffd640 0 client.25817 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 --2- 172.21.15.96:0/4025498403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451faa5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 >> 172.21.15.96:0/4025498403 conn(0x56451fa8ddc0 msgr2=0x56451fa8cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 shutdown_connections 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 1 -- 172.21.15.96:0/4025498403 wait complete. 2024-03-18T15:41:51.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.050+0000 7f4be8c34740 20 client.25817 trim_cache size 0 max 16384 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 Processor -- start 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 -- start start 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451fab7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a6d20 con 0x56451fa983b0 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a8980 con 0x56451fa93240 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4be8c34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a73f0 con 0x56451fa92850 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38816/0 (socket says 172.21.15.96:38816) 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55438/0 (socket says 172.21.15.96:55438) 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdeffd640 1 -- 172.21.15.96:0/3184177451 learned_addr learned my addr 172.21.15.96:0/3184177451 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.065+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451fab7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 -- 172.21.15.96:0/3184177451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 msgr2=0x56451fab7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451fab7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 -- 172.21.15.96:0/3184177451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 msgr2=0x56451fab84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 -- 172.21.15.96:0/3184177451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56451f7d57d0 con 0x56451fa93240 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdffff640 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ccb60 tx=0x7f4bd4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601b3baab769299c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bc57fa640 1 -- 172.21.15.96:0/3184177451 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd409a860 con 0x56451fa93240 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bc57fa640 1 -- 172.21.15.96:0/3184177451 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd40d0070 con 0x56451fa93240 2024-03-18T15:41:51.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.066+0000 7f4bc57fa640 1 -- 172.21.15.96:0/3184177451 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd409adf0 con 0x56451fa93240 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 -- 172.21.15.96:0/3184177451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 msgr2=0x56451faacac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ccb60 tx=0x7f4bd4000bc0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 -- 172.21.15.96:0/3184177451 shutdown_connections 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa983b0 0x56451fab84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa93240 0x56451faacac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 --2- 172.21.15.96:0/3184177451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa92850 0x56451fab7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 -- 172.21.15.96:0/3184177451 >> 172.21.15.96:0/3184177451 conn(0x56451fa8ddc0 msgr2=0x56451fa918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 -- 172.21.15.96:0/3184177451 shutdown_connections 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.067+0000 7f4be8c34740 1 -- 172.21.15.96:0/3184177451 wait complete. 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.068+0000 7f4be8c34740 1 -- start start 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.068+0000 7f4be8c34740 1 Processor -- start 2024-03-18T15:41:51.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 0x56451faac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 0x56451faaa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a81f0 con 0x56451fa93240 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a7b20 con 0x56451fa92850 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4be01a4d00 con 0x56451fa983b0 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4be8c34740 10 client.0 _ll_register_callbacks cb 0x7f4bd400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 0x56451faaa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4bdffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 0x56451faaa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:46800/0 (socket says 172.21.15.96:46800) 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4bdf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38828/0 (socket says 172.21.15.96:38828) 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.069+0000 7f4bdffff640 1 -- 172.21.15.96:0/1025114383 learned_addr learned my addr 172.21.15.96:0/1025114383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 0x56451faac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 -- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 msgr2=0x56451faaa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 0x56451faaa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 -- 172.21.15.96:0/1025114383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 msgr2=0x56451faac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 0x56451faac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 -- 172.21.15.96:0/1025114383 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56451faaaf40 con 0x56451fa92850 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 0x56451faac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bdf7fe640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bd002c8e0 tx=0x7f4bd002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3796e17a473324a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0054530 con 0x56451fa92850 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4bd00546d0 con 0x56451fa92850 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.070+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bd40d0070 con 0x56451fa92850 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4bd00046c0 con 0x56451fa92850 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 10 client.25823 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56451fb17fb0 con 0x56451fa92850 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bd0004ef0 con 0x56451fa92850 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f4bd005b070 con 0x56451fa92850 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 10 client.25823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 20 client.25823 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fb18940 0x56451fb1ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 4 client.25823 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4bd40d0070 con 0x56451fb18940 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.071+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fb18940 0x56451fb1ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.072+0000 7f4bbeffd640 10 client.25823 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:51.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.072+0000 7f4bdeffd640 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fb18940 0x56451fb1ad60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=947727196375bdb4 server_cookie=fd9ab16f77872f00 in_seq=0 out_seq=0 2024-03-18T15:41:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4bbeffd640 1 -- 172.21.15.96:0/1025114383 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4bd40d0070 con 0x56451fb18940 2024-03-18T15:41:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4bbeffd640 10 client.25823 handle_command_reply: tid=0 2024-03-18T15:41:51.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 client.25823 shutdown 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 msgr2=0x56451faac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bd002c8e0 tx=0x7f4bd002bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 shutdown_connections 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x56451fb18940 0x56451fb1ad60 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56451fa983b0 0x56451faaa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56451fa93240 0x56451faac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 --2- 172.21.15.96:0/1025114383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56451fa92850 0x56451faac2a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4bbeffd640 0 client.25823 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.078+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 >> 172.21.15.96:0/1025114383 conn(0x56451fa8ddc0 msgr2=0x56451fa8c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 shutdown_connections 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be8c34740 1 -- 172.21.15.96:0/1025114383 wait complete. 2024-03-18T15:41:51.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be8c34740 20 client.25823 trim_cache size 0 max 16384 2024-03-18T15:41:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 msgr2=0x7f4bc80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 0x7f4bc80a4d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0005ab0 tx=0x7f4bd002e040 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 msgr2=0x7f4be0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400e9f0 tx=0x7f4bd400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 shutdown_connections 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4bc80a2880 0x7f4bc80a4d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4be01022d0 0x7f4be019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4be0101910 0x7f4be0198130 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 --2- 172.21.15.96:0/2002099304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4be0100720 0x7f4be0197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.079+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 >> 172.21.15.96:0/2002099304 conn(0x7f4be00fbc90 msgr2=0x7f4be00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.080+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 shutdown_connections 2024-03-18T15:41:51.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.080+0000 7f4be5f76640 1 -- 172.21.15.96:0/2002099304 wait complete. 2024-03-18T15:41:51.092 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:41:51.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- 172.21.15.96:0/2124973763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 msgr2=0x7fe6c8101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- 172.21.15.96:0/2124973763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8101c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009860 tx=0x7fe6bc02bb70 comp rx=0 tx=0).stop 2024-03-18T15:41:51.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- 172.21.15.96:0/2124973763 shutdown_connections 2024-03-18T15:41:51.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- 172.21.15.96:0/2124973763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c81033c0 0x7fe6c8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- 172.21.15.96:0/2124973763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c8102a00 0x7fe6c8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- 172.21.15.96:0/2124973763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8101c30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- 172.21.15.96:0/2124973763 >> 172.21.15.96:0/2124973763 conn(0x7fe6c80fcd80 msgr2=0x7fe6c80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- 172.21.15.96:0/2124973763 shutdown_connections 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- 172.21.15.96:0/2124973763 wait complete. 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 Processor -- start 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- start start 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8102a00 0x7fe6c81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c810d490 con 0x7fe6c81033c0 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c810d310 con 0x7fe6c8101810 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cecea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c810d610 con 0x7fe6c8102a00 2024-03-18T15:41:51.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cd287640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.200+0000 7fe6cd287640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55468/0 (socket says 172.21.15.96:55468) 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 -- 172.21.15.96:0/3626133310 learned_addr learned my addr 172.21.15.96:0/3626133310 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8102a00 0x7fe6c81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cca86640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 -- 172.21.15.96:0/3626133310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8102a00 msgr2=0x7fe6c81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8102a00 0x7fe6c81980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 -- 172.21.15.96:0/3626133310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 msgr2=0x7fe6c8197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 -- 172.21.15.96:0/3626133310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c819f780 con 0x7fe6c81033c0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cd287640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000eb10 tx=0x7fe6c000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6c5ffb640 1 -- 172.21.15.96:0/3626133310 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c000ce20 con 0x7fe6c81033c0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c819fa70 con 0x7fe6c81033c0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6c5ffb640 1 -- 172.21.15.96:0/3626133310 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0005020 con 0x7fe6c81033c0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6c5ffb640 1 -- 172.21.15.96:0/3626133310 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0010690 con 0x7fe6c81033c0 2024-03-18T15:41:51.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.201+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c81a0010 con 0x7fe6c81033c0 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.204+0000 7fe6d19b2740 1 Processor -- start 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.204+0000 7fe6c5ffb640 1 -- 172.21.15.96:0/3626133310 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe6c0005540 con 0x7fe6c81033c0 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.204+0000 7fe6d19b2740 1 -- start start 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6b9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55596a616490 con 0x55596a6b1850 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55596a5bec90 con 0x55596a6b73b0 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55596a612530 con 0x55596a6b2240 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6cca86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55470/0 (socket says 172.21.15.96:55470) 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 -- 172.21.15.96:0/1458359738 learned_addr learned my addr 172.21.15.96:0/1458359738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c5ffb640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 0x7fe6b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 -- 172.21.15.96:0/1458359738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 msgr2=0x55596a6b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 -- 172.21.15.96:0/1458359738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 msgr2=0x55596a6b9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6b9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 -- 172.21.15.96:0/1458359738 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596a679500 con 0x55596a6b1850 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6cd287640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 0x7fe6b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c5ffb640 1 -- 172.21.15.96:0/3626133310 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe6c00cf150 con 0x7fe6c81033c0 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4013640 tx=0x7fe6b4009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437a97a1bd08c4ff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6b2ffd640 1 -- 172.21.15.96:0/1458359738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004050 con 0x55596a6b1850 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6b2ffd640 1 -- 172.21.15.96:0/1458359738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40041f0 con 0x55596a6b1850 2024-03-18T15:41:51.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.205+0000 7fe6b2ffd640 1 -- 172.21.15.96:0/1458359738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400f7e0 con 0x55596a6b1850 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1458359738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 msgr2=0x55596a6b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6cd287640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 0x7fe6b80a4cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0097f20 tx=0x7fe6c00667d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4013640 tx=0x7fe6b4009a10 comp rx=0 tx=0).stop 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1458359738 shutdown_connections 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6b9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1458359738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b1c70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1458359738 >> 172.21.15.96:0/1458359738 conn(0x55596a6acdc0 msgr2=0x55596a6ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1458359738 shutdown_connections 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.206+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1458359738 wait complete. 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 -- start start 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 Processor -- start 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6bc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55596a6ba360 con 0x55596a6b1850 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55596a5d2230 con 0x55596a6b73b0 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55596a6ba4e0 con 0x55596a6b2240 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6d19b2740 10 client.0 _ll_register_callbacks cb 0x55596a60e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55474/0 (socket says 172.21.15.96:55474) 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 -- 172.21.15.96:0/662828526 learned_addr learned my addr 172.21.15.96:0/662828526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6cca86640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6bc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 -- 172.21.15.96:0/662828526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 msgr2=0x55596a6bc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6bc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 -- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 msgr2=0x55596a6c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.207+0000 7fe6c7fff640 1 -- 172.21.15.96:0/662828526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596a5e1100 con 0x55596a6b1850 2024-03-18T15:41:51.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009530 tx=0x7fe6b4003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207d600a0c638584 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4009d70 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b40041f0 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe6b400f580 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6b400f780 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 10 client.16044 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55596a32a3f0 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b40041f0 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe6b401fa60 con 0x55596a6b1850 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 10 client.16044 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 20 client.16044 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a6c78a0 0x55596a6c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 4 client.16044 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:51.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55596a6239d0 con 0x55596a6c78a0 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.208+0000 7fe6cca86640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a6c78a0 0x55596a6c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.209+0000 7fe6cca86640 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a6c78a0 0x55596a6c9cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ceba320e09730696 server_cookie=52851e8bf91dd3b6 in_seq=0 out_seq=0 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.209+0000 7fe6a7fff640 10 client.16044 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.209+0000 7fe6a7fff640 1 -- 172.21.15.96:0/662828526 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55596a6239d0 con 0x55596a6c78a0 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.209+0000 7fe6a7fff640 10 client.16044 handle_command_reply: tid=0 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.209+0000 7fe6d19b2740 1 client.16044 shutdown 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 msgr2=0x55596a6bbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009530 tx=0x7fe6b4003be0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 shutdown_connections 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a6c78a0 0x55596a6c9cb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b2240 0x55596a6bc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6a7fff640 0 client.16044 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:51.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/662828526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6bbd50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 >> 172.21.15.96:0/662828526 conn(0x55596a6acdc0 msgr2=0x55596a6abfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 shutdown_connections 2024-03-18T15:41:51.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 1 -- 172.21.15.96:0/662828526 wait complete. 2024-03-18T15:41:51.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.210+0000 7fe6d19b2740 20 client.16044 trim_cache size 0 max 16384 2024-03-18T15:41:51.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 Processor -- start 2024-03-18T15:41:51.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 -- start start 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b73b0 0x55596a6c3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81a8e90 con 0x55596a6b1850 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81aaaf0 con 0x55596a6b2240 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.223+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81a9560 con 0x55596a6b73b0 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38878/0 (socket says 172.21.15.96:38878) 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 -- 172.21.15.96:0/1608521299 learned_addr learned my addr 172.21.15.96:0/1608521299 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b73b0 0x55596a6c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 -- 172.21.15.96:0/1608521299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b73b0 msgr2=0x55596a6c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b73b0 0x55596a6c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 -- 172.21.15.96:0/1608521299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 msgr2=0x55596a6b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 -- 172.21.15.96:0/1608521299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55596a3f47d0 con 0x55596a6b2240 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6cca86640 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009860 tx=0x7fe6bc034ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a23bbf1d781866b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6b27fc640 1 -- 172.21.15.96:0/1608521299 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc04f070 con 0x55596a6b2240 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6b27fc640 1 -- 172.21.15.96:0/1608521299 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004040 con 0x55596a6b2240 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.224+0000 7fe6b27fc640 1 -- 172.21.15.96:0/1608521299 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc039040 con 0x55596a6b2240 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1608521299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 msgr2=0x55596a6b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009860 tx=0x7fe6bc034ad0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1608521299 shutdown_connections 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b73b0 0x55596a6c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b2240 0x55596a6b3c20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/1608521299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b1850 0x55596a6b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1608521299 >> 172.21.15.96:0/1608521299 conn(0x55596a6acdc0 msgr2=0x55596a6bbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1608521299 shutdown_connections 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- 172.21.15.96:0/1608521299 wait complete. 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 -- start start 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.225+0000 7fe6d19b2740 1 Processor -- start 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b1850 0x55596a6c0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81aa360 con 0x55596a6b2240 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81a9c90 con 0x55596a6b73b0 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81a6e70 con 0x55596a6b1850 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6d19b2740 10 client.0 _ll_register_callbacks cb 0x7fe6b4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b1850 0x55596a6c0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55484/0 (socket says 172.21.15.96:55484) 2024-03-18T15:41:51.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 -- 172.21.15.96:0/3886932234 learned_addr learned my addr 172.21.15.96:0/3886932234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 -- 172.21.15.96:0/3886932234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b1850 msgr2=0x55596a6c0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b1850 0x55596a6c0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 -- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 msgr2=0x55596a6c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 -- 172.21.15.96:0/3886932234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596a6c1f90 con 0x55596a6b2240 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6cca86640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02e010 tx=0x7fe6bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f809529c799929 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.226+0000 7fe6a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc058060 con 0x55596a6b2240 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe6bc0041f0 con 0x55596a6b2240 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc004040 con 0x55596a6b2240 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc04f050 con 0x55596a6b2240 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 10 client.16050 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55596a6c3290 con 0x55596a6b2240 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004040 con 0x55596a6b2240 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe6bc0327d0 con 0x55596a6b2240 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 10 client.16050 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 20 client.16050 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a72dfb0 0x55596a7303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 4 client.16050 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55596a61dd20 con 0x55596a72dfb0 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.227+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a72dfb0 0x55596a7303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6a7fff640 10 client.16050 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6c7fff640 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a72dfb0 0x55596a7303d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88ca330f829608d6 server_cookie=793a8f06021f2e54 in_seq=0 out_seq=0 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6a7fff640 1 -- 172.21.15.96:0/3886932234 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55596a61dd20 con 0x55596a72dfb0 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6a7fff640 10 client.16050 handle_command_reply: tid=0 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 client.16050 shutdown 2024-03-18T15:41:51.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 msgr2=0x55596a6c0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc02e010 tx=0x7fe6bc000bc0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 shutdown_connections 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55596a72dfb0 0x55596a7303d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55596a6b73b0 0x55596a6c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55596a6b2240 0x55596a6c0990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 --2- 172.21.15.96:0/3886932234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55596a6b1850 0x55596a6c0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6a7fff640 0 client.16050 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 >> 172.21.15.96:0/3886932234 conn(0x55596a6acdc0 msgr2=0x55596a6bbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 shutdown_connections 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 1 -- 172.21.15.96:0/3886932234 wait complete. 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.228+0000 7fe6d19b2740 20 client.16050 trim_cache size 0 max 16384 2024-03-18T15:41:51.235 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 msgr2=0x7fe6b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 0x7fe6b80a4cc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0097f20 tx=0x7fe6c00667d0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 msgr2=0x7fe6c819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000eb10 tx=0x7fe6c000ef10 comp rx=0 tx=0).stop 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 shutdown_connections 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe6b80a2800 0x7fe6b80a4cc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6c81033c0 0x7fe6c819f110 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6c8102a00 0x7fe6c81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 --2- 172.21.15.96:0/3626133310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c8101810 0x7fe6c8197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 >> 172.21.15.96:0/3626133310 conn(0x7fe6c80fcd80 msgr2=0x7fe6c8105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 shutdown_connections 2024-03-18T15:41:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.229+0000 7fe6cecea640 1 -- 172.21.15.96:0/3626133310 wait complete. 2024-03-18T15:41:51.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:41:51.328 INFO:tasks.workunit.client.0.smithi096.stdout: 35 659486 259 482571 159 356190 570 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 -- 172.21.15.96:0/774100379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141018f0 msgr2=0x7fe414101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 --2- 172.21.15.96:0/774100379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141018f0 0x7fe414101d70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe408009900 tx=0x7fe40802bc30 comp rx=0 tx=0).stop 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 -- 172.21.15.96:0/774100379 shutdown_connections 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 --2- 172.21.15.96:0/774100379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4141022b0 0x7fe414108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 --2- 172.21.15.96:0/774100379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141018f0 0x7fe414101d70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 --2- 172.21.15.96:0/774100379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe414100700 0x7fe414100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 -- 172.21.15.96:0/774100379 >> 172.21.15.96:0/774100379 conn(0x7fe4140fbc90 msgr2=0x7fe4140fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 -- 172.21.15.96:0/774100379 shutdown_connections 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.347+0000 7fe418885640 1 -- 172.21.15.96:0/774100379 wait complete. 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 Processor -- start 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 -- start start 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414100700 0x7fe41406a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 0x7fe41419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe41410c360 con 0x7fe4141022b0 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe41410c1e0 con 0x7fe4141018f0 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe418885640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe41410c4e0 con 0x7fe414100700 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe41259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414100700 0x7fe41406a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55498/0 (socket says 172.21.15.96:55498) 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 0x7fe41419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.348+0000 7fe412d9d640 1 -- 172.21.15.96:0/3237472742 learned_addr learned my addr 172.21.15.96:0/3237472742 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 -- 172.21.15.96:0/3237472742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414100700 msgr2=0x7fe41406a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414100700 0x7fe41406a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 -- 172.21.15.96:0/3237472742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 msgr2=0x7fe41419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 0x7fe41419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 -- 172.21.15.96:0/3237472742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41419f720 con 0x7fe4141022b0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 0x7fe41419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe412d9d640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00eaa0 tx=0x7fe40c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe4037fe640 1 -- 172.21.15.96:0/3237472742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00cdb0 con 0x7fe4141022b0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe4037fe640 1 -- 172.21.15.96:0/3237472742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c005020 con 0x7fe4141022b0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41419fa10 con 0x7fe4141022b0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe4037fe640 1 -- 172.21.15.96:0/3237472742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0116b0 con 0x7fe4141022b0 2024-03-18T15:41:51.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.349+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41419fee0 con 0x7fe4141022b0 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.351+0000 7fe4037fe640 1 -- 172.21.15.96:0/3237472742 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe40c005560 con 0x7fe4141022b0 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 Processor -- start 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 -- start start 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe4037fe640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 0x7fe3fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec48fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec4903b0 0x5583ec492840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583ec3ef490 con 0x5583ec4903b0 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583ec397c90 con 0x5583ec48a850 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41b47c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ec3eb530 con 0x5583ec48b240 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 0x7fe3fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec4903b0 0x5583ec492840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38906/0 (socket says 172.21.15.96:38906) 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe4037fe640 1 -- 172.21.15.96:0/3237472742 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe40c0cec40 con 0x7fe4141022b0 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe411d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec48fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 -- 172.21.15.96:0/3158106473 learned_addr learned my addr 172.21.15.96:0/3158106473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 -- 172.21.15.96:0/3158106473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 msgr2=0x5583ec48fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec48fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 -- 172.21.15.96:0/3158106473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec4903b0 msgr2=0x5583ec492840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec4903b0 0x5583ec492840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.352+0000 7fe41259c640 1 -- 172.21.15.96:0/3158106473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ec452500 con 0x5583ec48a850 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 0x7fe3fc0a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe40802c7b0 tx=0x7fe40802c6a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe41259c640 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe404009ca0 tx=0x7fe404011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928bb709db0f68c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe400ff9640 1 -- 172.21.15.96:0/3158106473 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404004220 con 0x5583ec48a850 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe400ff9640 1 -- 172.21.15.96:0/3158106473 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4040043c0 con 0x5583ec48a850 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe400ff9640 1 -- 172.21.15.96:0/3158106473 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4040188e0 con 0x5583ec48a850 2024-03-18T15:41:51.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe41b47c740 1 -- 172.21.15.96:0/3158106473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 msgr2=0x5583ec48ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.353+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe404009ca0 tx=0x7fe404011810 comp rx=0 tx=0).stop 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- 172.21.15.96:0/3158106473 shutdown_connections 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec4903b0 0x5583ec492840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec48fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3158106473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec48a850 0x5583ec48ac70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- 172.21.15.96:0/3158106473 >> 172.21.15.96:0/3158106473 conn(0x5583ec485dc0 msgr2=0x5583ec4861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- 172.21.15.96:0/3158106473 shutdown_connections 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- 172.21.15.96:0/3158106473 wait complete. 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- start start 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 Processor -- start 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec495260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583ec493360 con 0x5583ec48a850 2024-03-18T15:41:51.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583ec3ab230 con 0x5583ec4903b0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583ec4934e0 con 0x5583ec48b240 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41b47c740 10 client.0 _ll_register_callbacks cb 0x5583ec3e7010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55510/0 (socket says 172.21.15.96:55510) 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 -- 172.21.15.96:0/176503459 learned_addr learned my addr 172.21.15.96:0/176503459 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe412d9d640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe411d9b640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec495260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 -- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 msgr2=0x5583ec495260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec495260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.354+0000 7fe41259c640 1 -- 172.21.15.96:0/176503459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 msgr2=0x5583ec49c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41259c640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41259c640 1 -- 172.21.15.96:0/176503459 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ec3ba100 con 0x5583ec48a850 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41259c640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe404011d10 tx=0x7fe404005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9eb145709795a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404027020 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe4040035f0 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4040043c0 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe404032050 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 10 client.16062 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583ec1033f0 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4040043c0 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe404004a90 con 0x5583ec48a850 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 10 client.16062 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 20 client.16062 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec4a0a40 0x5583ec4a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 4 client.16062 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.355+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ec3fc9d0 con 0x5583ec4a0a40 2024-03-18T15:41:51.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.356+0000 7fe411d9b640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec4a0a40 0x5583ec4a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.356+0000 7fe3f9ffb640 10 client.16062 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.356+0000 7fe411d9b640 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec4a0a40 0x5583ec4a2ea0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e70913f070a07b server_cookie=e300a1aaf0ecb5c4 in_seq=0 out_seq=0 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/176503459 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583ec3fc9d0 con 0x5583ec4a0a40 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe3f9ffb640 10 client.16062 handle_command_reply: tid=0 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 client.16062 shutdown 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 msgr2=0x5583ec494d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe404011d10 tx=0x7fe404005270 comp rx=0 tx=0).stop 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 shutdown_connections 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec495260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec4a0a40 0x5583ec4a2ea0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 --2- 172.21.15.96:0/176503459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec494d20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe3f9ffb640 0 client.16062 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:51.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 >> 172.21.15.96:0/176503459 conn(0x5583ec485dc0 msgr2=0x5583ec484fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 shutdown_connections 2024-03-18T15:41:51.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 1 -- 172.21.15.96:0/176503459 wait complete. 2024-03-18T15:41:51.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.357+0000 7fe41b47c740 20 client.16062 trim_cache size 0 max 16384 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 Processor -- start 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 -- start start 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48a850 0x5583ec4951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec484750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141a8d80 con 0x5583ec48b240 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141aa9e0 con 0x5583ec4903b0 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41b47c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141a9450 con 0x5583ec48a850 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55526/0 (socket says 172.21.15.96:55526) 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 -- 172.21.15.96:0/966753377 learned_addr learned my addr 172.21.15.96:0/966753377 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe41259c640 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48a850 0x5583ec4951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe412d9d640 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec484750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 -- 172.21.15.96:0/966753377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48a850 msgr2=0x5583ec4951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48a850 0x5583ec4951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 -- 172.21.15.96:0/966753377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 msgr2=0x5583ec484750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec484750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 -- 172.21.15.96:0/966753377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ec1cd7d0 con 0x5583ec48b240 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.371+0000 7fe411d9b640 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe40802c710 tx=0x7fe408049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7837361f02f33d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe3fbfff640 1 -- 172.21.15.96:0/966753377 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408059070 con 0x5583ec48b240 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe3fbfff640 1 -- 172.21.15.96:0/966753377 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408053070 con 0x5583ec48b240 2024-03-18T15:41:51.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe3fbfff640 1 -- 172.21.15.96:0/966753377 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408004e00 con 0x5583ec48b240 2024-03-18T15:41:51.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 -- 172.21.15.96:0/966753377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 msgr2=0x5583ec49c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe40802c710 tx=0x7fe408049000 comp rx=0 tx=0).stop 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 -- 172.21.15.96:0/966753377 shutdown_connections 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec484750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48b240 0x5583ec49c220 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 --2- 172.21.15.96:0/966753377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48a850 0x5583ec4951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.372+0000 7fe41b47c740 1 -- 172.21.15.96:0/966753377 >> 172.21.15.96:0/966753377 conn(0x5583ec485dc0 msgr2=0x5583ec494ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- 172.21.15.96:0/966753377 shutdown_connections 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- 172.21.15.96:0/966753377 wait complete. 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- start start 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 Processor -- start 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec499ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141aa250 con 0x5583ec48a850 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141a9b80 con 0x5583ec4903b0 2024-03-18T15:41:51.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4141a6d60 con 0x5583ec48b240 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41b47c740 10 client.0 _ll_register_callbacks cb 0x7fe40c005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55532/0 (socket says 172.21.15.96:55532) 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 -- 172.21.15.96:0/3386885785 learned_addr learned my addr 172.21.15.96:0/3386885785 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe412d9d640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec499ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 -- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 msgr2=0x5583ec499ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec499ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 -- 172.21.15.96:0/3386885785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 msgr2=0x5583ec49aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.373+0000 7fe41259c640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41259c640 1 -- 172.21.15.96:0/3386885785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ec49b0e0 con 0x5583ec48a850 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41259c640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4040059f0 tx=0x7fe4040098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486619dc2cc05e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404027040 con 0x5583ec48a850 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe404018b70 con 0x5583ec48a850 2024-03-18T15:41:51.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe408053070 con 0x5583ec48a850 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe40401e040 con 0x5583ec48a850 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 10 client.16068 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583ec49c3e0 con 0x5583ec48a850 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408053070 con 0x5583ec48a850 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe404004650 con 0x5583ec48a850 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 10 client.16068 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 20 client.16068 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec507540 0x5583ec509960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 4 client.16068 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.374+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583ec3f6d20 con 0x5583ec507540 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec507540 0x5583ec509960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe411d9b640 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec507540 0x5583ec509960 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10cedb50b63bc3e server_cookie=988184e15c7235a9 in_seq=0 out_seq=0 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe3f9ffb640 10 client.16068 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:51.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe3f9ffb640 1 -- 172.21.15.96:0/3386885785 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583ec3f6d20 con 0x5583ec507540 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe3f9ffb640 10 client.16068 handle_command_reply: tid=0 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 client.16068 shutdown 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 msgr2=0x5583ec4995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4040059f0 tx=0x7fe4040098d0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 shutdown_connections 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ec4903b0 0x5583ec49aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583ec48b240 0x5583ec499ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583ec507540 0x5583ec509960 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 --2- 172.21.15.96:0/3386885785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ec48a850 0x5583ec4995a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe3f9ffb640 0 client.16068 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 >> 172.21.15.96:0/3386885785 conn(0x5583ec485dc0 msgr2=0x5583ec489020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 shutdown_connections 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 1 -- 172.21.15.96:0/3386885785 wait complete. 2024-03-18T15:41:51.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.375+0000 7fe41b47c740 20 client.16068 trim_cache size 0 max 16384 2024-03-18T15:41:51.383 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 msgr2=0x7fe3fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 0x7fe3fc0a4cc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe40802c7b0 tx=0x7fe40802c6a0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 msgr2=0x7fe41419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00eaa0 tx=0x7fe40c00ef60 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 shutdown_connections 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe3fc0a2800 0x7fe3fc0a4cc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4141022b0 0x7fe41419ee60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4141018f0 0x7fe41419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 --2- 172.21.15.96:0/3237472742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe414100700 0x7fe41406a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 >> 172.21.15.96:0/3237472742 conn(0x7fe4140fbc90 msgr2=0x7fe414104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 shutdown_connections 2024-03-18T15:41:51.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:51.376+0000 7fe418885640 1 -- 172.21.15.96:0/3237472742 wait complete. 2024-03-18T15:41:52.388 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:41:52.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:41:52.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- 172.21.15.96:0/2467426535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a40ffe60 msgr2=0x7fe2a4100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/2467426535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4100280 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe29400aa10 tx=0x7fe29402cf70 comp rx=0 tx=0).stop 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- 172.21.15.96:0/2467426535 shutdown_connections 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/2467426535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a4101970 0x7fe2a4108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/2467426535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 0x7fe2a4101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/2467426535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4100280 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- 172.21.15.96:0/2467426535 >> 172.21.15.96:0/2467426535 conn(0x7fe2a40fb3d0 msgr2=0x7fe2a40fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- 172.21.15.96:0/2467426535 shutdown_connections 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- 172.21.15.96:0/2467426535 wait complete. 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 Processor -- start 2024-03-18T15:41:52.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.502+0000 7fe2a95b3640 1 -- start start 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 0x7fe2a4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410b260 con 0x7fe2a4101970 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410b0e0 con 0x7fe2a4100fb0 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410b3e0 con 0x7fe2a40ffe60 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55550/0 (socket says 172.21.15.96:55550) 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 -- 172.21.15.96:0/3515794941 learned_addr learned my addr 172.21.15.96:0/3515794941 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 0x7fe2a4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 -- 172.21.15.96:0/3515794941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffe60 msgr2=0x7fe2a4197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 -- 172.21.15.96:0/3515794941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 msgr2=0x7fe2a4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 0x7fe2a4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 -- 172.21.15.96:0/3515794941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a419f770 con 0x7fe2a4101970 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009fd0 tx=0x7fe29c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe28bfff640 1 -- 172.21.15.96:0/3515794941 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c017070 con 0x7fe2a4101970 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe28bfff640 1 -- 172.21.15.96:0/3515794941 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c003e50 con 0x7fe2a4101970 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a419fa60 con 0x7fe2a4101970 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe28bfff640 1 -- 172.21.15.96:0/3515794941 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0106c0 con 0x7fe2a4101970 2024-03-18T15:41:52.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.503+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a419ffb0 con 0x7fe2a4101970 2024-03-18T15:41:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe28bfff640 1 -- 172.21.15.96:0/3515794941 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe29c003970 con 0x7fe2a4101970 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 Processor -- start 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 -- start start 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe28bfff640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 0x7fe28c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e54305e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54308980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e542650d0 con 0x560e54300990 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e5420dc70 con 0x560e54301380 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2ac199740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560e54261170 con 0x560e543064f0 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 0x7fe28c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54308980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55560/0 (socket says 172.21.15.96:55560) 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2648584759 learned_addr learned my addr 172.21.15.96:0/2648584759 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e54305e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.506+0000 7fe28bfff640 1 -- 172.21.15.96:0/3515794941 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe29c0ce670 con 0x7fe2a4101970 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2648584759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 msgr2=0x560e54308980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54308980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2648584759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 msgr2=0x560e54305e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e54305e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2648584759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e540417e0 con 0x560e54300990 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54308980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 0x7fe28c0a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b6e0 tx=0x7fe29800bbd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe294005ab0 tx=0x7fe294005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf7c1f4b1e24677 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2897fa640 1 -- 172.21.15.96:0/2648584759 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294043070 con 0x560e54300990 2024-03-18T15:41:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2897fa640 1 -- 172.21.15.96:0/2648584759 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294035640 con 0x560e54300990 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.507+0000 7fe2897fa640 1 -- 172.21.15.96:0/2648584759 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2940044d0 con 0x560e54300990 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- 172.21.15.96:0/2648584759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 msgr2=0x560e54300db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe294005ab0 tx=0x7fe294005dd0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- 172.21.15.96:0/2648584759 shutdown_connections 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54308980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e54305e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2648584759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e54300db0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- 172.21.15.96:0/2648584759 >> 172.21.15.96:0/2648584759 conn(0x560e542c8c50 msgr2=0x560e542fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- 172.21.15.96:0/2648584759 shutdown_connections 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- 172.21.15.96:0/2648584759 wait complete. 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- start start 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 Processor -- start 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e5430b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e543122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e54309440 con 0x560e54300990 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560e542210d0 con 0x560e54301380 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560e543095c0 con 0x560e543064f0 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2ac199740 10 client.0 _ll_register_callbacks cb 0x560e541ea760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55572/0 (socket says 172.21.15.96:55572) 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2671397489 learned_addr learned my addr 172.21.15.96:0/2671397489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e5430b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.508+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e543122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 msgr2=0x560e543122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e543122d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2671397489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 msgr2=0x560e5430b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e5430b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2671397489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e540a1f70 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e5430b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe294037040 tx=0x7fe294004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39a8d69e10e1bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29404c070 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe2940048d0 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe294035640 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe294058050 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2ac199740 10 client.16080 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e53f39170 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294035640 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe29400b040 con 0x560e54300990 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.509+0000 7fe2ac199740 10 client.16080 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2ac199740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2ac199740 20 client.16080 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54316480 0x560e543188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2ac199740 4 client.16080 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e54272610 con 0x560e54316480 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54316480 0x560e543188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54316480 0x560e543188e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9aae89ea0a569 server_cookie=9e027ce172a1eb4e in_seq=0 out_seq=0 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.510+0000 7fe286ff5640 10 client.16080 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe286ff5640 1 -- 172.21.15.96:0/2671397489 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560e54272610 con 0x560e54316480 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe286ff5640 10 client.16080 handle_command_reply: tid=0 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 client.16080 shutdown 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 msgr2=0x560e5430ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe294037040 tx=0x7fe294004610 comp rx=0 tx=0).stop 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 shutdown_connections 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e543122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54301380 0x560e5430b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54316480 0x560e543188e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2671397489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54300990 0x560e5430ad50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 >> 172.21.15.96:0/2671397489 conn(0x560e542c8c50 msgr2=0x560e542fb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe286ff5640 0 client.16080 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 shutdown_connections 2024-03-18T15:41:52.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.511+0000 7fe2ac199740 1 -- 172.21.15.96:0/2671397489 wait complete. 2024-03-18T15:41:52.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.512+0000 7fe2ac199740 20 client.16080 trim_cache size 0 max 16384 2024-03-18T15:41:52.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 Processor -- start 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 -- start start 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 0x560e542fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54312290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41a8d40 con 0x560e54301380 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41aa9a0 con 0x560e54300990 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2ac199740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41a9410 con 0x560e543064f0 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.529+0000 7fe2a27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 0x560e542fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38950/0 (socket says 172.21.15.96:38950) 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2260801856 learned_addr learned my addr 172.21.15.96:0/2260801856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54312290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2260801856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 msgr2=0x560e54312290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54312290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2260801856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 msgr2=0x560e542fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 0x560e542fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/2260801856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e54302e20 con 0x560e54300990 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 0x560e542fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe294000c00 tx=0x7fe2940058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2fa5de5da6e9b43 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe288ff9640 1 -- 172.21.15.96:0/2260801856 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29404c030 con 0x560e54300990 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe288ff9640 1 -- 172.21.15.96:0/2260801856 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29400d000 con 0x560e54300990 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.530+0000 7fe288ff9640 1 -- 172.21.15.96:0/2260801856 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294043070 con 0x560e54300990 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- 172.21.15.96:0/2260801856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 msgr2=0x560e542fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe294000c00 tx=0x7fe2940058a0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- 172.21.15.96:0/2260801856 shutdown_connections 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e543064f0 0x560e54312290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e54301380 0x560e542fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- 172.21.15.96:0/2260801856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e542fb360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- 172.21.15.96:0/2260801856 >> 172.21.15.96:0/2260801856 conn(0x560e542c8c50 msgr2=0x560e542fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- 172.21.15.96:0/2260801856 shutdown_connections 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- 172.21.15.96:0/2260801856 wait complete. 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- start start 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 Processor -- start 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e54301380 0x560e5430fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41aa210 con 0x560e543064f0 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41a9b40 con 0x560e54300990 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41a6d20 con 0x560e54301380 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2ac199740 10 client.0 _ll_register_callbacks cb 0x7fe29400a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.531+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38952/0 (socket says 172.21.15.96:38952) 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/571135931 learned_addr learned my addr 172.21.15.96:0/571135931 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55600/0 (socket says 172.21.15.96:55600) 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e54301380 0x560e5430fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e54301380 msgr2=0x560e5430fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e54301380 0x560e5430fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/571135931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 msgr2=0x560e54310ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 -- 172.21.15.96:0/571135931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e54311190 con 0x560e54300990 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a37fe640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2a2ffd640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe29400a7d0 tx=0x7fe294004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba3efcfc91ad22b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294043070 con 0x560e54300990 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe294004f90 con 0x560e54300990 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29400d000 con 0x560e54300990 2024-03-18T15:41:52.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe29404c040 con 0x560e54300990 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2ac199740 10 client.25877 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e543124d0 con 0x560e54300990 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.532+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29400d000 con 0x560e54300990 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe294003930 con 0x560e54300990 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 10 client.25877 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 20 client.25877 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54385550 0x560e54387970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 4 client.25877 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e5430a190 con 0x560e54385550 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54385550 0x560e54387970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe286ff5640 10 client.25877 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.533+0000 7fe2a27fc640 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54385550 0x560e54387970 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e8e7de31fc526a server_cookie=bb1d54b198d45e94 in_seq=0 out_seq=0 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.534+0000 7fe286ff5640 1 -- 172.21.15.96:0/571135931 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e5430a190 con 0x560e54385550 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.534+0000 7fe286ff5640 10 client.25877 handle_command_reply: tid=0 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.534+0000 7fe2ac199740 1 client.25877 shutdown 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 msgr2=0x560e5430f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe29400a7d0 tx=0x7fe294004610 comp rx=0 tx=0).stop 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 shutdown_connections 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560e54385550 0x560e54387970 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e543064f0 0x560e54310ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560e54301380 0x560e5430fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 --2- 172.21.15.96:0/571135931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e54300990 0x560e5430f6a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe286ff5640 0 client.25877 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 >> 172.21.15.96:0/571135931 conn(0x560e542c8c50 msgr2=0x560e542fb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 shutdown_connections 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 1 -- 172.21.15.96:0/571135931 wait complete. 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2ac199740 20 client.25877 trim_cache size 0 max 16384 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "210876af-c1c5-487a-839c-dd82471180fb", 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:41:52.542 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 msgr2=0x7fe28c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 0x7fe28c0a4d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe29800b6e0 tx=0x7fe29800bbd0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 msgr2=0x7fe2a419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009fd0 tx=0x7fe29c00b260 comp rx=0 tx=0).stop 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 shutdown_connections 2024-03-18T15:41:52.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe28c0a2880 0x7fe28c0a4d40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2a4101970 0x7fe2a419f100 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2a4100fb0 0x7fe2a4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 --2- 172.21.15.96:0/3515794941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2a40ffe60 0x7fe2a4197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 >> 172.21.15.96:0/3515794941 conn(0x7fe2a40fb3d0 msgr2=0x7fe2a41043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 shutdown_connections 2024-03-18T15:41:52.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.535+0000 7fe2a95b3640 1 -- 172.21.15.96:0/3515794941 wait complete. 2024-03-18T15:41:52.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:41:52.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- 172.21.15.96:0/3976288745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681033a0 msgr2=0x7fe068109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 --2- 172.21.15.96:0/3976288745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681033a0 0x7fe068109c30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe060009ca0 tx=0x7fe06002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- 172.21.15.96:0/3976288745 shutdown_connections 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 --2- 172.21.15.96:0/3976288745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681033a0 0x7fe068109c30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 --2- 172.21.15.96:0/3976288745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681029e0 0x7fe068102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 --2- 172.21.15.96:0/3976288745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681017f0 0x7fe068101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- 172.21.15.96:0/3976288745 >> 172.21.15.96:0/3976288745 conn(0x7fe0680fcd40 msgr2=0x7fe0680ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- 172.21.15.96:0/3976288745 shutdown_connections 2024-03-18T15:41:52.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- 172.21.15.96:0/3976288745 wait complete. 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 Processor -- start 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.654+0000 7fe06f705640 1 -- start start 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681017f0 0x7fe068197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681033a0 0x7fe06819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe06810d470 con 0x7fe0681029e0 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe06810d2f0 con 0x7fe0681033a0 2024-03-18T15:41:52.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe06810d5f0 con 0x7fe0681017f0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55624/0 (socket says 172.21.15.96:55624) 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 -- 172.21.15.96:0/2232080223 learned_addr learned my addr 172.21.15.96:0/2232080223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681017f0 0x7fe068197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06dca2640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681033a0 0x7fe06819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 -- 172.21.15.96:0/2232080223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681017f0 msgr2=0x7fe068197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681017f0 0x7fe068197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 -- 172.21.15.96:0/2232080223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681033a0 msgr2=0x7fe06819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681033a0 0x7fe06819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 -- 172.21.15.96:0/2232080223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06819f6e0 con 0x7fe0681029e0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06cca0640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e9f0 tx=0x7fe05800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe05e7fc640 1 -- 172.21.15.96:0/2232080223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800cd80 con 0x7fe0681029e0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06819f9d0 con 0x7fe0681029e0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.655+0000 7fe05e7fc640 1 -- 172.21.15.96:0/2232080223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe058005500 con 0x7fe0681029e0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.656+0000 7fe05e7fc640 1 -- 172.21.15.96:0/2232080223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058010690 con 0x7fe0681029e0 2024-03-18T15:41:52.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.656+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06819ff20 con 0x7fe0681029e0 2024-03-18T15:41:52.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.658+0000 7fe05e7fc640 1 -- 172.21.15.96:0/2232080223 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe0580046c0 con 0x7fe0681029e0 2024-03-18T15:41:52.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.658+0000 7fe05e7fc640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 0x7fe0540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.659+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 0x7fe0540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.659+0000 7fe05e7fc640 1 -- 172.21.15.96:0/2232080223 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe0580cee80 con 0x7fe0681029e0 2024-03-18T15:41:52.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.659+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 0x7fe0540a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe068102840 tx=0x7fe064005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 Processor -- start 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 -- start start 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d89c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d7fd500 con 0x55563d89cf70 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55563d7a5be0 con 0x55563d8a20e0 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe0723c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55563d7f95a0 con 0x55563d89c580 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d89c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55634/0 (socket says 172.21.15.96:55634) 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38970/0 (socket says 172.21.15.96:38970) 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06cca0640 1 -- 172.21.15.96:0/962306025 learned_addr learned my addr 172.21.15.96:0/962306025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 -- 172.21.15.96:0/962306025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 msgr2=0x55563d8a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 -- 172.21.15.96:0/962306025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 msgr2=0x55563d8a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06d4a1640 1 -- 172.21.15.96:0/962306025 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563d7b94b0 con 0x55563d89c580 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06dca2640 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.660+0000 7fe06cca0640 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d89c9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe06400e560 tx=0x7fe064014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38586ae70843d8de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe0537fe640 1 -- 172.21.15.96:0/962306025 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064004020 con 0x55563d89c580 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe0537fe640 1 -- 172.21.15.96:0/962306025 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0640041c0 con 0x55563d89c580 2024-03-18T15:41:52.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe0537fe640 1 -- 172.21.15.96:0/962306025 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06401c670 con 0x55563d89c580 2024-03-18T15:41:52.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe0723c5740 1 -- 172.21.15.96:0/962306025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 msgr2=0x55563d89c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.661+0000 7fe0723c5740 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d89c9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe06400e560 tx=0x7fe064014000 comp rx=0 tx=0).stop 2024-03-18T15:41:52.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- 172.21.15.96:0/962306025 shutdown_connections 2024-03-18T15:41:52.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d8a20e0 0x55563d8a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89cf70 0x55563d8a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- 172.21.15.96:0/962306025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d89c9a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- 172.21.15.96:0/962306025 >> 172.21.15.96:0/962306025 conn(0x55563d8936e0 msgr2=0x55563d893ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- 172.21.15.96:0/962306025 shutdown_connections 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- 172.21.15.96:0/962306025 wait complete. 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- start start 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 Processor -- start 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d8a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d8a5090 con 0x55563d8a20e0 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55563d7b9160 con 0x55563d89cf70 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55563d8a5210 con 0x55563d89c580 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe0723c5740 10 client.0 _ll_register_callbacks cb 0x55563d79d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.662+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d8a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d8a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:47032/0 (socket says 172.21.15.96:47032) 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06d4a1640 1 -- 172.21.15.96:0/3348219189 learned_addr learned my addr 172.21.15.96:0/3348219189 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06cca0640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 -- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 msgr2=0x55563d8a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d8a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 -- 172.21.15.96:0/3348219189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 msgr2=0x55563d8a6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8a6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 -- 172.21.15.96:0/3348219189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563d5db7d0 con 0x55563d8a20e0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06cca0640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe06dca2640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8adfe0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe06002c630 tx=0x7fe060009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285ce9e4b68c3640 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe050ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060044070 con 0x55563d8a20e0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe060032690 con 0x55563d8a20e0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0640041c0 con 0x55563d8a20e0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe060050050 con 0x55563d8a20e0 2024-03-18T15:41:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe0723c5740 10 client.16098 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.663+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55563d5113f0 con 0x55563d8a20e0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0640041c0 con 0x55563d8a20e0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe0600040d0 con 0x55563d8a20e0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 10 client.16098 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 20 client.16098 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d8b2650 0x55563d8b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 4 client.16098 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55563d80aa40 con 0x55563d8b2650 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.664+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d8b2650 0x55563d8b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.665+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d8b2650 0x55563d8b4a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb447493a9743c10 server_cookie=6c439667e182695c in_seq=0 out_seq=0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.665+0000 7fe050ff9640 10 client.16098 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.665+0000 7fe050ff9640 1 -- 172.21.15.96:0/3348219189 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55563d80aa40 con 0x55563d8b2650 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.665+0000 7fe050ff9640 10 client.16098 handle_command_reply: tid=0 2024-03-18T15:41:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.665+0000 7fe0723c5740 1 client.16098 shutdown 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 msgr2=0x55563d8adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8adfe0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe06002c630 tx=0x7fe060009800 comp rx=0 tx=0).stop 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 shutdown_connections 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d8b2650 0x55563d8b4a90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8adfe0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe050ff9640 0 client.16098 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8a6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 --2- 172.21.15.96:0/3348219189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d8a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 >> 172.21.15.96:0/3348219189 conn(0x55563d8936e0 msgr2=0x55563d892900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 shutdown_connections 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 1 -- 172.21.15.96:0/3348219189 wait complete. 2024-03-18T15:41:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.666+0000 7fe0723c5740 20 client.16098 trim_cache size 0 max 16384 2024-03-18T15:41:52.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:52 smithi096 ceph-mon[31930]: pgmap v405: 97 pgs: 97 active+clean; 148 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 Processor -- start 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 -- start start 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d892780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a6ab0 con 0x55563d8a20e0 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a8710 con 0x55563d89cf70 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe0723c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a7180 con 0x55563d89c580 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d892780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55644/0 (socket says 172.21.15.96:55644) 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d892780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:47034/0 (socket says 172.21.15.96:47034) 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06dca2640 1 -- 172.21.15.96:0/1706606545 learned_addr learned my addr 172.21.15.96:0/1706606545 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 -- 172.21.15.96:0/1706606545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 msgr2=0x55563d892780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d892780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 -- 172.21.15.96:0/1706606545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 msgr2=0x55563d8a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.686+0000 7fe06cca0640 1 -- 172.21.15.96:0/1706606545 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563d4d10f0 con 0x55563d89cf70 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.687+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8adfa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe058097c50 tx=0x7fe058086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b1992f72bae244 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.687+0000 7fe052ffd640 1 -- 172.21.15.96:0/1706606545 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05809bef0 con 0x55563d89cf70 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.687+0000 7fe052ffd640 1 -- 172.21.15.96:0/1706606545 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05809b7f0 con 0x55563d89cf70 2024-03-18T15:41:52.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.687+0000 7fe052ffd640 1 -- 172.21.15.96:0/1706606545 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580afe30 con 0x55563d89cf70 2024-03-18T15:41:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- 172.21.15.96:0/1706606545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 msgr2=0x55563d8adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8adfa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe058097c50 tx=0x7fe058086040 comp rx=0 tx=0).stop 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- 172.21.15.96:0/1706606545 shutdown_connections 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d8a20e0 0x55563d8a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8adfa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1706606545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d89c580 0x55563d892780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- 172.21.15.96:0/1706606545 >> 172.21.15.96:0/1706606545 conn(0x55563d8936e0 msgr2=0x55563d892070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- 172.21.15.96:0/1706606545 shutdown_connections 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- 172.21.15.96:0/1706606545 wait complete. 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 -- start start 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.688+0000 7fe0723c5740 1 Processor -- start 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 0x55563d8ab730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8abc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d8a20e0 0x55563d8acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a7f80 con 0x55563d89c580 2024-03-18T15:41:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a78b0 con 0x55563d89cf70 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681a4a90 con 0x55563d8a20e0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe0723c5740 10 client.0 _ll_register_callbacks cb 0x7fe058005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 0x55563d8ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06dca2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d8a20e0 0x55563d8acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06d4a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 0x55563d8ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55650/0 (socket says 172.21.15.96:55650) 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06d4a1640 1 -- 172.21.15.96:0/1586612997 learned_addr learned my addr 172.21.15.96:0/1586612997 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 -- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d8a20e0 msgr2=0x55563d8acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d8a20e0 0x55563d8acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 -- 172.21.15.96:0/1586612997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 msgr2=0x55563d8ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 0x55563d8ab730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.689+0000 7fe06cca0640 1 -- 172.21.15.96:0/1586612997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563d7c49d0 con 0x55563d89cf70 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe06cca0640 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8abc70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55563d80ab60 tx=0x7fe0580980b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2f4f2ed935b811 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:41:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580a3d00 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe05809ac40 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05809b7f0 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe05809f940 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 10 client.25895 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55563d9189d0 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05809b7f0 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fe0580bb040 con 0x55563d89cf70 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 10 client.25895 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 20 client.25895 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d9192c0 0x55563d91b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 4 client.25895 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.690+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55563d804d90 con 0x55563d9192c0 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d9192c0 0x55563d91b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe050ff9640 10 client.25895 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe06d4a1640 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d9192c0 0x55563d91b6e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd033371c654b5d server_cookie=f64999e2db097362 in_seq=0 out_seq=0 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe050ff9640 1 -- 172.21.15.96:0/1586612997 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55563d804d90 con 0x55563d9192c0 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe050ff9640 10 client.25895 handle_command_reply: tid=0 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 client.25895 shutdown 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 msgr2=0x55563d8abc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8abc70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55563d80ab60 tx=0x7fe0580980b0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 shutdown_connections 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55563d8a20e0 0x55563d8acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55563d89cf70 0x55563d8abc70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55563d9192c0 0x55563d91b6e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 --2- 172.21.15.96:0/1586612997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d89c580 0x55563d8ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe050ff9640 0 client.25895 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.691+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 >> 172.21.15.96:0/1586612997 conn(0x55563d8936e0 msgr2=0x55563d8971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 shutdown_connections 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe0723c5740 1 -- 172.21.15.96:0/1586612997 wait complete. 2024-03-18T15:41:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe0723c5740 20 client.25895 trim_cache size 0 max 16384 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "210876af-c1c5-487a-839c-dd82471180fb": { 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "210876af-c1c5-487a-839c-dd82471180fb", 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:41:52.698 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:41:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 msgr2=0x7fe0540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 0x7fe0540a4cc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe068102840 tx=0x7fe064005f90 comp rx=0 tx=0).stop 2024-03-18T15:41:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 msgr2=0x7fe068198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e9f0 tx=0x7fe05800edf0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 shutdown_connections 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe0540a2800 0x7fe0540a4cc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe0681033a0 0x7fe06819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0681029e0 0x7fe068198000 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 --2- 172.21.15.96:0/2232080223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0681017f0 0x7fe068197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 >> 172.21.15.96:0/2232080223 conn(0x7fe0680fcd40 msgr2=0x7fe068105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 shutdown_connections 2024-03-18T15:41:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:41:52.692+0000 7fe06f705640 1 -- 172.21.15.96:0/2232080223 wait complete. 2024-03-18T15:41:52.704 INFO:tasks.cephfs.filesystem:scrub status for tag:210876af-c1c5-487a-839c-dd82471180fb - {'path': '/', 'tag': '210876af-c1c5-487a-839c-dd82471180fb', 'options': 'recursive,force'} 2024-03-18T15:41:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:52 smithi137 ceph-mon[35868]: pgmap v405: 97 pgs: 97 active+clean; 148 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2024-03-18T15:41:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:52 smithi193 ceph-mon[36702]: pgmap v405: 97 pgs: 97 active+clean; 148 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 499 op/s 2024-03-18T15:41:53.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:41:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:53 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:41:53.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:53 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:41:53.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:53 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:41:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:54 smithi137 ceph-mon[35868]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 541 op/s 2024-03-18T15:41:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:54 smithi193 ceph-mon[36702]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 541 op/s 2024-03-18T15:41:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:54 smithi096 ceph-mon[31930]: pgmap v406: 97 pgs: 97 active+clean; 145 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.8 MiB/s wr, 541 op/s 2024-03-18T15:41:56.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:56 smithi137 ceph-mon[35868]: pgmap v407: 97 pgs: 97 active+clean; 150 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-03-18T15:41:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:56 smithi193 ceph-mon[36702]: pgmap v407: 97 pgs: 97 active+clean; 150 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-03-18T15:41:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:56 smithi096 ceph-mon[31930]: pgmap v407: 97 pgs: 97 active+clean; 150 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 463 op/s 2024-03-18T15:41:58.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:41:58 smithi137 ceph-mon[35868]: pgmap v408: 97 pgs: 97 active+clean; 150 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-03-18T15:41:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:41:58 smithi193 ceph-mon[36702]: pgmap v408: 97 pgs: 97 active+clean; 150 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-03-18T15:41:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:41:58 smithi096 ceph-mon[31930]: pgmap v408: 97 pgs: 97 active+clean; 150 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-03-18T15:41:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.286+0000 7f0f5ac97640 1 -- 172.21.15.96:0/1543698907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f541022b0 msgr2=0x7f0f54108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.286+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/1543698907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f541022b0 0x7f0f54108b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ca0 tx=0x7f0f4c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.286+0000 7f0f5ac97640 1 -- 172.21.15.96:0/1543698907 shutdown_connections 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.286+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/1543698907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f541022b0 0x7f0f54108b40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/1543698907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541018f0 0x7f0f54101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/1543698907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f54100700 0x7f0f54100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- 172.21.15.96:0/1543698907 >> 172.21.15.96:0/1543698907 conn(0x7f0f540fbc70 msgr2=0x7f0f540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- 172.21.15.96:0/1543698907 shutdown_connections 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- 172.21.15.96:0/1543698907 wait complete. 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 Processor -- start 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- start start 2024-03-18T15:41:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 0x7f0f54196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541022b0 0x7f0f5419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410c380 con 0x7f0f54100700 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410c200 con 0x7f0f541018f0 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f5ac97640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5410c500 con 0x7f0f541022b0 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57560/0 (socket says 172.21.15.96:57560) 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f58a33640 1 -- 172.21.15.96:0/2895385667 learned_addr learned my addr 172.21.15.96:0/2895385667 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.287+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 0x7f0f54196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f59234640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541022b0 0x7f0f5419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 -- 172.21.15.96:0/2895385667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541022b0 msgr2=0x7f0f5419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541022b0 0x7f0f5419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 -- 172.21.15.96:0/2895385667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 msgr2=0x7f0f54196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 0x7f0f54196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 -- 172.21.15.96:0/2895385667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5419e550 con 0x7f0f54100700 2024-03-18T15:41:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 0x7f0f54196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f58a33640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f51ffb640 1 -- 172.21.15.96:0/2895385667 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800cd80 con 0x7f0f54100700 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5419e840 con 0x7f0f54100700 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f51ffb640 1 -- 172.21.15.96:0/2895385667 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f480035f0 con 0x7f0f54100700 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f51ffb640 1 -- 172.21.15.96:0/2895385667 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48010690 con 0x7f0f54100700 2024-03-18T15:41:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.288+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5419ed90 con 0x7f0f54100700 2024-03-18T15:41:59.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f51ffb640 1 -- 172.21.15.96:0/2895385667 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0f480037a0 con 0x7f0f54100700 2024-03-18T15:41:59.298 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 Processor -- start 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 -- start start 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f51ffb640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 0x7f0f440a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a32830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a32ea0 0x561ab2a35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ab2992a40 con 0x561ab2a2d340 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561ab293b920 con 0x561ab2a2dd30 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.291+0000 7f0f5d883740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ab298eae0 con 0x561ab2a32ea0 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f59234640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 0x7f0f440a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57568/0 (socket says 172.21.15.96:57568) 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f58a33640 1 -- 172.21.15.96:0/2206310291 learned_addr learned my addr 172.21.15.96:0/2206310291 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:59.299 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a32830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f51ffb640 1 -- 172.21.15.96:0/2895385667 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0f480cee40 con 0x7f0f54100700 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f59234640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a32ea0 0x561ab2a35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 -- 172.21.15.96:0/2206310291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a32ea0 msgr2=0x561ab2a35330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a32ea0 0x561ab2a35330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 -- 172.21.15.96:0/2206310291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 msgr2=0x561ab2a2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 -- 172.21.15.96:0/2206310291 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab2953460 con 0x561ab2a2dd30 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f58a33640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f59234640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 0x7f0f440a4d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c02c630 tx=0x7f0f4c009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f53fff640 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a32830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009fa0 tx=0x7f0f400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509c2b9337722ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f3effd640 1 -- 172.21.15.96:0/2206310291 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004310 con 0x561ab2a2dd30 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f3effd640 1 -- 172.21.15.96:0/2206310291 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f400044b0 con 0x561ab2a2dd30 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.292+0000 7f0f3effd640 1 -- 172.21.15.96:0/2206310291 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40003610 con 0x561ab2a2dd30 2024-03-18T15:41:59.300 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- 172.21.15.96:0/2206310291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 msgr2=0x561ab2a32830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a32830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009fa0 tx=0x7f0f400118d0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- 172.21.15.96:0/2206310291 shutdown_connections 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a32ea0 0x561ab2a35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a32830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 --2- 172.21.15.96:0/2206310291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- 172.21.15.96:0/2206310291 >> 172.21.15.96:0/2206310291 conn(0x561ab2a288b0 msgr2=0x561ab2a28cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- 172.21.15.96:0/2206310291 shutdown_connections 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- 172.21.15.96:0/2206310291 wait complete. 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 -- start start 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.293+0000 7f0f5d883740 1 Processor -- start 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2d340 0x561ab2a398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a39e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ab2a35e50 con 0x561ab2a32ea0 2024-03-18T15:41:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561ab294ed70 con 0x561ab2a2dd30 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ab2a35fd0 con 0x561ab2a2d340 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f5d883740 10 client.0 _ll_register_callbacks cb 0x561ab26a6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f53fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a39e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57578/0 (socket says 172.21.15.96:57578) 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 -- 172.21.15.96:0/887996819 learned_addr learned my addr 172.21.15.96:0/887996819 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f58a33640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2d340 0x561ab2a398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 -- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2d340 msgr2=0x561ab2a398f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2d340 0x561ab2a398f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 -- 172.21.15.96:0/887996819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 msgr2=0x561ab2a39e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a39e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 -- 172.21.15.96:0/887996819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab26a7210 con 0x561ab2a32ea0 2024-03-18T15:41:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f53fff640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a39e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f59234640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ca0 tx=0x7f0f4c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b51f148f428c9d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.294+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c039040 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0f4c004450 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f400044b0 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f4c004650 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 10 client.16116 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ab2773620 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f400044b0 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f0f4c03f040 con 0x561ab2a32ea0 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 10 client.16116 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 10 client.16116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 20 client.16116 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2a45830 0x561ab2a47cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 4 client.16116 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ab299ff80 con 0x561ab2a45830 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.295+0000 7f0f58a33640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2a45830 0x561ab2a47cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.296+0000 7f0f58a33640 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2a45830 0x561ab2a47cb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1849db42d41d2d server_cookie=6717c94f78e59179 in_seq=0 out_seq=0 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.296+0000 7f0f33fff640 10 client.16116 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.296+0000 7f0f33fff640 1 -- 172.21.15.96:0/887996819 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561ab299ff80 con 0x561ab2a45830 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.296+0000 7f0f33fff640 10 client.16116 handle_command_reply: tid=0 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.296+0000 7f0f5d883740 1 client.16116 shutdown 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 msgr2=0x561ab2a40e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009ca0 tx=0x7f0f4c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 shutdown_connections 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2a45830 0x561ab2a47cb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a40e00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f33fff640 0 client.16116 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2dd30 0x561ab2a39e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 --2- 172.21.15.96:0/887996819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2d340 0x561ab2a398f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 >> 172.21.15.96:0/887996819 conn(0x561ab2a288b0 msgr2=0x561ab2a27bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 shutdown_connections 2024-03-18T15:41:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 1 -- 172.21.15.96:0/887996819 wait complete. 2024-03-18T15:41:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.297+0000 7f0f5d883740 20 client.16116 trim_cache size 0 max 16384 2024-03-18T15:41:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.311+0000 7f0f5d883740 1 Processor -- start 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 -- start start 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a40dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 0x561ab2a27320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a6ad0 con 0x561ab2a2d340 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a8730 con 0x561ab2a32ea0 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f5d883740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a71a0 con 0x561ab2a2dd30 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57580/0 (socket says 172.21.15.96:57580) 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 -- 172.21.15.96:0/4187990602 learned_addr learned my addr 172.21.15.96:0/4187990602 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f53fff640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a40dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f59234640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 0x561ab2a27320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 -- 172.21.15.96:0/4187990602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 msgr2=0x561ab2a40dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a40dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 -- 172.21.15.96:0/4187990602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 msgr2=0x561ab2a27320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 0x561ab2a27320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 -- 172.21.15.96:0/4187990602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ab2a29620 con 0x561ab2a2d340 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f59234640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 0x561ab2a27320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f53fff640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a40dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f58a33640 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0f4809b4b0 tx=0x7f0f48086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931ef027dbd84604 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f3e7fc640 1 -- 172.21.15.96:0/4187990602 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48060070 con 0x561ab2a2d340 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f3e7fc640 1 -- 172.21.15.96:0/4187990602 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4809beb0 con 0x561ab2a2d340 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.312+0000 7f0f3e7fc640 1 -- 172.21.15.96:0/4187990602 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48099ab0 con 0x561ab2a2d340 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- 172.21.15.96:0/4187990602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 msgr2=0x561ab2a392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0f4809b4b0 tx=0x7f0f48086040 comp rx=0 tx=0).stop 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- 172.21.15.96:0/4187990602 shutdown_connections 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a32ea0 0x561ab2a27320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a40dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4187990602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a2d340 0x561ab2a392f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- 172.21.15.96:0/4187990602 >> 172.21.15.96:0/4187990602 conn(0x561ab2a288b0 msgr2=0x561ab2a38be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- 172.21.15.96:0/4187990602 shutdown_connections 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- 172.21.15.96:0/4187990602 wait complete. 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 -- start start 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.313+0000 7f0f5d883740 1 Processor -- start 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 0x561ab2a3cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a3d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a7fa0 con 0x561ab2a32ea0 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a78d0 con 0x561ab2a2d340 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f541a4ab0 con 0x561ab2a2dd30 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 10 client.0 _ll_register_callbacks cb 0x7f0f40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 0x561ab2a3cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f58a33640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 0x561ab2a3cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:51726/0 (socket says 172.21.15.96:51726) 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57588/0 (socket says 172.21.15.96:57588) 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f58a33640 1 -- 172.21.15.96:0/4047456735 learned_addr learned my addr 172.21.15.96:0/4047456735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f53fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a3d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 -- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 msgr2=0x561ab2a3d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a3d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 -- 172.21.15.96:0/4047456735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 msgr2=0x561ab2a3cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 0x561ab2a3cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 -- 172.21.15.96:0/4047456735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ab2a40730 con 0x561ab2a32ea0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f59234640 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c032e90 tx=0x7f0f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b5c479c815a4ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f33fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c05d070 con 0x561ab2a32ea0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4809beb0 con 0x561ab2a32ea0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0f4c004450 con 0x561ab2a32ea0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.314+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f4c069050 con 0x561ab2a32ea0 2024-03-18T15:41:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 10 client.16122 fetch_fsmap learned FSMap version 29 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561ab2aac280 con 0x561ab2a32ea0 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4809beb0 con 0x561ab2a32ea0 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f0f4c03f070 con 0x561ab2a32ea0 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 10 client.16122 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 20 client.16122 populate_metadata read hostname 'smithi096' 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2aacc60 0x561ab2aaf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 4 client.16122 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ab299a2d0 con 0x561ab2aacc60 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f58a33640 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2aacc60 0x561ab2aaf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f33fff640 10 client.16122 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.315+0000 7f0f58a33640 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2aacc60 0x561ab2aaf080 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbec5268e07eef04 server_cookie=d5f084b1eb70dd64 in_seq=0 out_seq=0 2024-03-18T15:41:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f33fff640 1 -- 172.21.15.96:0/4047456735 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ab299a2d0 con 0x561ab2aacc60 2024-03-18T15:41:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f33fff640 10 client.16122 handle_command_reply: tid=0 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 client.16122 shutdown 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 msgr2=0x561ab2a400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c032e90 tx=0x7f0f4c009bb0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 shutdown_connections 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561ab2aacc60 0x561ab2aaf080 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ab2a32ea0 0x561ab2a400f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ab2a2dd30 0x561ab2a3d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f33fff640 0 client.16122 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 --2- 172.21.15.96:0/4047456735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561ab2a2d340 0x561ab2a3cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 >> 172.21.15.96:0/4047456735 conn(0x561ab2a288b0 msgr2=0x561ab2a2bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 shutdown_connections 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 1 -- 172.21.15.96:0/4047456735 wait complete. 2024-03-18T15:41:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.323+0000 7f0f5d883740 20 client.16122 trim_cache size 0 max 16384 2024-03-18T15:41:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:41:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:41:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:41:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 msgr2=0x7f0f440a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 0x7f0f440a4d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c02c630 tx=0x7f0f4c009800 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 msgr2=0x7f0f54196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ee80 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 shutdown_connections 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0f440a28a0 0x7f0f440a4d60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f541022b0 0x7f0f5419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f541018f0 0x7f0f54196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 --2- 172.21.15.96:0/2895385667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f54100700 0x7f0f54196930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 >> 172.21.15.96:0/2895385667 conn(0x7f0f540fbc70 msgr2=0x7f0f54104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:41:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 shutdown_connections 2024-03-18T15:41:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:41:59.324+0000 7f0f5ac97640 1 -- 172.21.15.96:0/2895385667 wait complete. 2024-03-18T15:42:00.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:00 smithi137 ceph-mon[35868]: pgmap v409: 97 pgs: 97 active+clean; 154 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-18T15:42:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:00 smithi193 ceph-mon[36702]: pgmap v409: 97 pgs: 97 active+clean; 154 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-18T15:42:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:00 smithi096 ceph-mon[31930]: pgmap v409: 97 pgs: 97 active+clean; 154 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 217 op/s 2024-03-18T15:42:01.328 INFO:tasks.workunit.client.0.smithi096.stdout: 41 1092296 351 807272 413 588687 1000 2024-03-18T15:42:02.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:02 smithi137 ceph-mon[35868]: pgmap v410: 97 pgs: 97 active+clean; 154 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 909 KiB/s wr, 110 op/s 2024-03-18T15:42:02.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:02 smithi193 ceph-mon[36702]: pgmap v410: 97 pgs: 97 active+clean; 154 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 909 KiB/s wr, 110 op/s 2024-03-18T15:42:02.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:02 smithi096 ceph-mon[31930]: pgmap v410: 97 pgs: 97 active+clean; 154 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 909 KiB/s wr, 110 op/s 2024-03-18T15:42:04.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:04 smithi137 ceph-mon[35868]: pgmap v411: 97 pgs: 97 active+clean; 160 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-18T15:42:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:04 smithi193 ceph-mon[36702]: pgmap v411: 97 pgs: 97 active+clean; 160 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-18T15:42:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:04 smithi096 ceph-mon[31930]: pgmap v411: 97 pgs: 97 active+clean; 160 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-03-18T15:42:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:06 smithi137 ceph-mon[35868]: pgmap v412: 97 pgs: 97 active+clean; 160 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-03-18T15:42:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:06 smithi193 ceph-mon[36702]: pgmap v412: 97 pgs: 97 active+clean; 160 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-03-18T15:42:06.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:06 smithi096 ceph-mon[31930]: pgmap v412: 97 pgs: 97 active+clean; 160 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 81 op/s 2024-03-18T15:42:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:08 smithi193 ceph-mon[36702]: pgmap v413: 97 pgs: 97 active+clean; 160 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-03-18T15:42:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:08 smithi096 ceph-mon[31930]: pgmap v413: 97 pgs: 97 active+clean; 160 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-03-18T15:42:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:08 smithi137 ceph-mon[35868]: pgmap v413: 97 pgs: 97 active+clean; 160 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.1 MiB/s wr, 70 op/s 2024-03-18T15:42:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:10 smithi193 ceph-mon[36702]: pgmap v414: 97 pgs: 97 active+clean; 170 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-03-18T15:42:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:10 smithi096 ceph-mon[31930]: pgmap v414: 97 pgs: 97 active+clean; 170 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-03-18T15:42:11.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:10 smithi137 ceph-mon[35868]: pgmap v414: 97 pgs: 97 active+clean; 170 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-03-18T15:42:11.328 INFO:tasks.workunit.client.0.smithi096.stdout: 44 502820 204 381589 253 268977 850 2024-03-18T15:42:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1794360817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec105390 msgr2=0x7ff7ec105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1794360817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec105390 0x7ff7ec105810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1794360817 shutdown_connections 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1794360817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec105390 0x7ff7ec105810 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1794360817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1794360817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 0x7ff7ec1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1794360817 >> 172.21.15.96:0/1794360817 conn(0x7ff7ec0fbc70 msgr2=0x7ff7ec0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1794360817 shutdown_connections 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.416+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1794360817 wait complete. 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 Processor -- start 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 -- start start 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 0x7ff7ec19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec105390 0x7ff7ec1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec113a70 con 0x7ff7ec1029e0 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1138f0 con 0x7ff7ec105390 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f3f7e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec113bf0 con 0x7ff7ec100070 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48772/0 (socket says 172.21.15.96:48772) 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 -- 172.21.15.96:0/1615374594 learned_addr learned my addr 172.21.15.96:0/1615374594 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 0x7ff7ec19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f251b640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec105390 0x7ff7ec1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 -- 172.21.15.96:0/1615374594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 msgr2=0x7ff7ec19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 0x7ff7ec19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 -- 172.21.15.96:0/1615374594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec105390 msgr2=0x7ff7ec1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec105390 0x7ff7ec1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.417+0000 7ff7f1519640 1 -- 172.21.15.96:0/1615374594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec1a5be0 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7f1519640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7e2ffd640 1 -- 172.21.15.96:0/1615374594 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00cd80 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7e2ffd640 1 -- 172.21.15.96:0/1615374594 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7dc010910 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec1a5ed0 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7e2ffd640 1 -- 172.21.15.96:0/1615374594 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc018990 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.418+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1a6390 con 0x7ff7ec1029e0 2024-03-18T15:42:12.423 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.420+0000 7ff7e2ffd640 1 -- 172.21.15.96:0/1615374594 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff7dc010ac0 con 0x7ff7ec1029e0 2024-03-18T15:42:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.420+0000 7ff7f6b6a740 1 Processor -- start 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.420+0000 7ff7f6b6a740 1 -- start start 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 0x5587b2ac5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7e2ffd640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 0x7ff7d80a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac5ea0 0x5587b2ac8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587b2a25a40 con 0x5587b2ac0340 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5587b29ce920 con 0x5587b2ac0d30 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587b2a21ae0 con 0x5587b2ac5ea0 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f1519640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 0x7ff7d80a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48786/0 (socket says 172.21.15.96:48786) 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f1d1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 0x5587b2ac5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 -- 172.21.15.96:0/4231287548 learned_addr learned my addr 172.21.15.96:0/4231287548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac5ea0 0x5587b2ac8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7e2ffd640 1 -- 172.21.15.96:0/1615374594 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff7dc0cedc0 con 0x7ff7ec1029e0 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 -- 172.21.15.96:0/4231287548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac5ea0 msgr2=0x5587b2ac8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac5ea0 0x5587b2ac8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 -- 172.21.15.96:0/4231287548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 msgr2=0x5587b2ac5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 0x5587b2ac5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 -- 172.21.15.96:0/4231287548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b29e6460 con 0x5587b2ac0340 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 0x5587b2ac5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f1519640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 0x7ff7d80a4d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc09b770 tx=0x7ff7dc06b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10e413899821a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7d7fff640 1 -- 172.21.15.96:0/4231287548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4048070 con 0x5587b2ac0340 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7d7fff640 1 -- 172.21.15.96:0/4231287548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e4003970 con 0x5587b2ac0340 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.421+0000 7ff7d7fff640 1 -- 172.21.15.96:0/4231287548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4039040 con 0x5587b2ac0340 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4231287548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 msgr2=0x5587b2ac0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e4009360 comp rx=0 tx=0).stop 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4231287548 shutdown_connections 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac5ea0 0x5587b2ac8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0d30 0x5587b2ac5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4231287548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2ac0760 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4231287548 >> 172.21.15.96:0/4231287548 conn(0x5587b2abb8b0 msgr2=0x5587b2abbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4231287548 shutdown_connections 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4231287548 wait complete. 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.422+0000 7ff7f6b6a740 1 -- start start 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 Processor -- start 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2aca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5587b2ac8df0 con 0x5587b2ac5ea0 2024-03-18T15:42:12.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5587b29e1d70 con 0x5587b2ac0340 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587b2ac8f70 con 0x5587b2ac0d30 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f6b6a740 10 client.0 _ll_register_callbacks cb 0x5587b2739a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48788/0 (socket says 172.21.15.96:48788) 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 -- 172.21.15.96:0/99148580 learned_addr learned my addr 172.21.15.96:0/99148580 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f1d1a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2aca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 -- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 msgr2=0x5587b2acac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 -- 172.21.15.96:0/99148580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 msgr2=0x5587b2aca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2aca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.423+0000 7ff7f251b640 1 -- 172.21.15.96:0/99148580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b273a210 con 0x5587b2ac5ea0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f251b640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4032420 tx=0x7ff7e40099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f73dc9ec189aa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4048040 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff7e4032890 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e4003970 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7e4039040 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587b2806620 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e4003970 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7ff7e404e040 con 0x5587b2ac5ea0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 20 client.16134 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2ad60e0 0x5587b2ad84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 4 client.16134 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.424+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587b2a32f80 con 0x5587b2ad60e0 2024-03-18T15:42:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.425+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2ad60e0 0x5587b2ad84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.425+0000 7ff7d57fa640 10 client.16134 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.425+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2ad60e0 0x5587b2ad84d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b216d6ed2a97a82 server_cookie=9283d56850528b8a in_seq=0 out_seq=0 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7d57fa640 1 -- 172.21.15.96:0/99148580 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5587b2a32f80 con 0x5587b2ad60e0 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7d57fa640 10 client.16134 handle_command_reply: tid=0 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 client.16134 shutdown 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 msgr2=0x5587b2ad1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4032420 tx=0x7ff7e40099f0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 shutdown_connections 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2ad60e0 0x5587b2ad84d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad1bf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7d57fa640 0 client.16134 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/99148580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2aca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 >> 172.21.15.96:0/99148580 conn(0x5587b2abb8b0 msgr2=0x5587b2ababb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 shutdown_connections 2024-03-18T15:42:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.426+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/99148580 wait complete. 2024-03-18T15:42:12.437 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.427+0000 7ff7f6b6a740 20 client.16134 trim_cache size 0 max 16384 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.441+0000 7ff7f6b6a740 1 Processor -- start 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.441+0000 7ff7f6b6a740 1 -- start start 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2ad1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac5ea0 0x5587b2aba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1af2e0 con 0x5587b2ac0340 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1b0f40 con 0x5587b2ac5ea0 2024-03-18T15:42:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1af9b0 con 0x5587b2ac0d30 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48804/0 (socket says 172.21.15.96:48804) 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 -- 172.21.15.96:0/482903328 learned_addr learned my addr 172.21.15.96:0/482903328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2ad1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac5ea0 0x5587b2aba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 -- 172.21.15.96:0/482903328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 msgr2=0x5587b2ad1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2ad1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 -- 172.21.15.96:0/482903328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac5ea0 msgr2=0x5587b2aba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac5ea0 0x5587b2aba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 -- 172.21.15.96:0/482903328 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587b2ad8840 con 0x5587b2ac0340 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7f251b640 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e4036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5858206ca21287ba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7d77fe640 1 -- 172.21.15.96:0/482903328 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4048070 con 0x5587b2ac0340 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7d77fe640 1 -- 172.21.15.96:0/482903328 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e4003970 con 0x5587b2ac0340 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.442+0000 7ff7d77fe640 1 -- 172.21.15.96:0/482903328 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4004000 con 0x5587b2ac0340 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/482903328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 msgr2=0x5587b2aba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e4036040 comp rx=0 tx=0).stop 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/482903328 shutdown_connections 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac5ea0 0x5587b2aba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2ad1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/482903328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac0340 0x5587b2aba320 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/482903328 >> 172.21.15.96:0/482903328 conn(0x5587b2abb8b0 msgr2=0x5587b2ac20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/482903328 shutdown_connections 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/482903328 wait complete. 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 -- start start 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.443+0000 7ff7f6b6a740 1 Processor -- start 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2acf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1b07b0 con 0x5587b2ac5ea0 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1b00e0 con 0x5587b2ac0340 2024-03-18T15:42:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1ad2c0 con 0x5587b2ac0d30 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 10 client.0 _ll_register_callbacks cb 0x7ff7e4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48818/0 (socket says 172.21.15.96:48818) 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 -- 172.21.15.96:0/4218661194 learned_addr learned my addr 172.21.15.96:0/4218661194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2acf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 -- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 msgr2=0x5587b2acfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 -- 172.21.15.96:0/4218661194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 msgr2=0x5587b2acf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2acf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 -- 172.21.15.96:0/4218661194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587b2ad1390 con 0x5587b2ac5ea0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f251b640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005d70 tx=0x7ff7e4045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2007ec4b505f613 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4048020 con 0x5587b2ac5ea0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e4003970 con 0x5587b2ac5ea0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.444+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff7e4003fc0 con 0x5587b2ac5ea0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7e4042480 con 0x5587b2ac5ea0 2024-03-18T15:42:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 10 client.16140 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587b2b3c870 con 0x5587b2ac5ea0 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e4003970 con 0x5587b2ac5ea0 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7ff7e403f050 con 0x5587b2ac5ea0 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 20 client.16140 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2b3d000 0x5587b2b3f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 4 client.16140 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587b2a2d2d0 con 0x5587b2b3d000 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.445+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2b3d000 0x5587b2b3f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.446+0000 7ff7f1d1a640 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2b3d000 0x5587b2b3f420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa1a4cb5c10349c server_cookie=40d96f2202bbdc3 in_seq=0 out_seq=0 2024-03-18T15:42:12.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.446+0000 7ff7d57fa640 10 client.16140 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.452+0000 7ff7d57fa640 1 -- 172.21.15.96:0/4218661194 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587b2a2d2d0 con 0x5587b2b3d000 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.452+0000 7ff7d57fa640 10 client.16140 handle_command_reply: tid=0 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.452+0000 7ff7f6b6a740 1 client.16140 shutdown 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 msgr2=0x5587b2ad0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005d70 tx=0x7ff7e4045040 comp rx=0 tx=0).stop 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 shutdown_connections 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5587b2b3d000 0x5587b2b3f420 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5587b2ac5ea0 0x5587b2ad0da0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587b2ac0d30 0x5587b2acfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 --2- 172.21.15.96:0/4218661194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5587b2ac0340 0x5587b2acf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7d57fa640 0 client.16140 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 >> 172.21.15.96:0/4218661194 conn(0x5587b2abb8b0 msgr2=0x5587b2abeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 shutdown_connections 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 1 -- 172.21.15.96:0/4218661194 wait complete. 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f6b6a740 20 client.16140 trim_cache size 0 max 16384 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:42:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:42:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 msgr2=0x7ff7d80a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 0x7ff7d80a4d60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc09b770 tx=0x7ff7dc06b3c0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 msgr2=0x7ff7ec19e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 shutdown_connections 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff7d80a28a0 0x7ff7d80a4d60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7ec105390 0x7ff7ec1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7ec1029e0 0x7ff7ec19e500 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 --2- 172.21.15.96:0/1615374594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7ec100070 0x7ff7ec19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.453+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 >> 172.21.15.96:0/1615374594 conn(0x7ff7ec0fbc70 msgr2=0x7ff7ec1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.454+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 shutdown_connections 2024-03-18T15:42:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:12.454+0000 7ff7f3f7e640 1 -- 172.21.15.96:0/1615374594 wait complete. 2024-03-18T15:42:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:12 smithi193 ceph-mon[36702]: pgmap v415: 97 pgs: 97 active+clean; 184 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-03-18T15:42:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:12 smithi096 ceph-mon[31930]: pgmap v415: 97 pgs: 97 active+clean; 184 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-03-18T15:42:13.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:12 smithi137 ceph-mon[35868]: pgmap v415: 97 pgs: 97 active+clean; 184 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 230 op/s 2024-03-18T15:42:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:14 smithi193 ceph-mon[36702]: pgmap v416: 97 pgs: 97 active+clean; 211 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 413 op/s 2024-03-18T15:42:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:14 smithi096 ceph-mon[31930]: pgmap v416: 97 pgs: 97 active+clean; 211 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 413 op/s 2024-03-18T15:42:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:14 smithi137 ceph-mon[35868]: pgmap v416: 97 pgs: 97 active+clean; 211 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 413 op/s 2024-03-18T15:42:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:16 smithi193 ceph-mon[36702]: pgmap v417: 97 pgs: 97 active+clean; 222 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-03-18T15:42:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:16 smithi096 ceph-mon[31930]: pgmap v417: 97 pgs: 97 active+clean; 222 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-03-18T15:42:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:16 smithi137 ceph-mon[35868]: pgmap v417: 97 pgs: 97 active+clean; 222 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-03-18T15:42:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:18 smithi193 ceph-mon[36702]: pgmap v418: 97 pgs: 97 active+clean; 221 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-03-18T15:42:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:18 smithi096 ceph-mon[31930]: pgmap v418: 97 pgs: 97 active+clean; 221 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-03-18T15:42:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:18 smithi137 ceph-mon[35868]: pgmap v418: 97 pgs: 97 active+clean; 221 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.8 MiB/s wr, 471 op/s 2024-03-18T15:42:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:20 smithi193 ceph-mon[36702]: pgmap v419: 97 pgs: 97 active+clean; 225 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-03-18T15:42:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:20 smithi096 ceph-mon[31930]: pgmap v419: 97 pgs: 97 active+clean; 225 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-03-18T15:42:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:20 smithi137 ceph-mon[35868]: pgmap v419: 97 pgs: 97 active+clean; 225 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.4 MiB/s wr, 500 op/s 2024-03-18T15:42:21.328 INFO:tasks.workunit.client.0.smithi096.stdout: 50 1067956 449 824073 361 595876 878 2024-03-18T15:42:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:21 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:21.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:21 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:22.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:21 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:22.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:42:22.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- 172.21.15.96:0/2261258316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1033a0 msgr2=0x7f435c109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 --2- 172.21.15.96:0/2261258316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1033a0 0x7f435c109c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4354009ca0 tx=0x7f435402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- 172.21.15.96:0/2261258316 shutdown_connections 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 --2- 172.21.15.96:0/2261258316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1033a0 0x7f435c109c30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 --2- 172.21.15.96:0/2261258316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1029e0 0x7f435c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 --2- 172.21.15.96:0/2261258316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1017f0 0x7f435c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- 172.21.15.96:0/2261258316 >> 172.21.15.96:0/2261258316 conn(0x7f435c0fcd80 msgr2=0x7f435c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- 172.21.15.96:0/2261258316 shutdown_connections 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- 172.21.15.96:0/2261258316 wait complete. 2024-03-18T15:42:22.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 Processor -- start 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.821+0000 7f43623df640 1 -- start start 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1017f0 0x7f435c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1033a0 0x7f435c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c10d470 con 0x7f435c1029e0 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f435c10d2f0 con 0x7f435c1033a0 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f435c10d5f0 con 0x7f435c1017f0 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33542/0 (socket says 172.21.15.96:33542) 2024-03-18T15:42:22.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 -- 172.21.15.96:0/1479747246 learned_addr learned my addr 172.21.15.96:0/1479747246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435bfff640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1017f0 0x7f435c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f436097c640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1033a0 0x7f435c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 -- 172.21.15.96:0/1479747246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1017f0 msgr2=0x7f435c197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1017f0 0x7f435c197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 -- 172.21.15.96:0/1479747246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1033a0 msgr2=0x7f435c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1033a0 0x7f435c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 -- 172.21.15.96:0/1479747246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c19f6d0 con 0x7f435c1029e0 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f435000e9f0 tx=0x7f435000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43597fa640 1 -- 172.21.15.96:0/1479747246 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000cd80 con 0x7f435c1029e0 2024-03-18T15:42:22.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c19f9c0 con 0x7f435c1029e0 2024-03-18T15:42:22.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43597fa640 1 -- 172.21.15.96:0/1479747246 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350005500 con 0x7f435c1029e0 2024-03-18T15:42:22.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43597fa640 1 -- 172.21.15.96:0/1479747246 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350010690 con 0x7f435c1029e0 2024-03-18T15:42:22.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.822+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c19ff10 con 0x7f435c1029e0 2024-03-18T15:42:22.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 Processor -- start 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f43597fa640 1 -- 172.21.15.96:0/1479747246 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f43500046c0 con 0x7f435c1029e0 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 -- start start 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 0x564adbf4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4bb60 0x564adbf50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564adbeb0500 con 0x564adbf50cd0 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564adbe58be0 con 0x564adbf4b170 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.825+0000 7f4364fc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564adbeac5a0 con 0x564adbf4bb60 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33544/0 (socket says 172.21.15.96:33544) 2024-03-18T15:42:22.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4bb60 0x564adbf50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 -- 172.21.15.96:0/1053238638 learned_addr learned my addr 172.21.15.96:0/1053238638 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f436097c640 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 0x564adbf4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 -- 172.21.15.96:0/1053238638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4bb60 msgr2=0x564adbf50660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4bb60 0x564adbf50660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 -- 172.21.15.96:0/1053238638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 msgr2=0x564adbf4b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 0x564adbf4b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 -- 172.21.15.96:0/1053238638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adbe6c4b0 con 0x564adbf50cd0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f436097c640 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 0x564adbf4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f43597fa640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 0x7f43440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f436097c640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 0x7f43440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f435b7fe640 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43500d2de0 tx=0x7f43500d2ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeb45adf4adf581 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f43597fa640 1 -- 172.21.15.96:0/1479747246 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f43500cee60 con 0x7f435c1029e0 2024-03-18T15:42:22.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f43427fc640 1 -- 172.21.15.96:0/1053238638 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350097a70 con 0x564adbf50cd0 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f43427fc640 1 -- 172.21.15.96:0/1053238638 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435009bed0 con 0x564adbf50cd0 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.826+0000 7f43427fc640 1 -- 172.21.15.96:0/1053238638 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435009b130 con 0x564adbf50cd0 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- 172.21.15.96:0/1053238638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 msgr2=0x564adbf53160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f436097c640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 0x7f43440a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4354005e30 tx=0x7f4354032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43500d2de0 tx=0x7f43500d2ed0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- 172.21.15.96:0/1053238638 shutdown_connections 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf50cd0 0x564adbf53160 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4bb60 0x564adbf50660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 --2- 172.21.15.96:0/1053238638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4b170 0x564adbf4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- 172.21.15.96:0/1053238638 >> 172.21.15.96:0/1053238638 conn(0x564adbf466e0 msgr2=0x564adbf46ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- 172.21.15.96:0/1053238638 shutdown_connections 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- 172.21.15.96:0/1053238638 wait complete. 2024-03-18T15:42:22.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.827+0000 7f4364fc9740 1 -- start start 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 Processor -- start 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564adbf53c20 con 0x564adbf4b170 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564adbe6c160 con 0x564adbf4bb60 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564adbf53da0 con 0x564adbf50cd0 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f4364fc9740 10 client.0 _ll_register_callbacks cb 0x564adbe50150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33560/0 (socket says 172.21.15.96:33560) 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 -- 172.21.15.96:0/376547048 learned_addr learned my addr 172.21.15.96:0/376547048 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:22.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f436097c640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 -- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 msgr2=0x564adbf5cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 -- 172.21.15.96:0/376547048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 msgr2=0x564adbf55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 -- 172.21.15.96:0/376547048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adbc8e7d0 con 0x564adbf4b170 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f435bfff640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f434c012700 tx=0x7f434c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa42692c3306a68d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.828+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c00f8c0 con 0x564adbf4b170 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f434c00fa60 con 0x564adbf4b170 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f435009bed0 con 0x564adbf4b170 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f434c00fc60 con 0x564adbf4b170 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 10 client.16152 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:22.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564adbbc43f0 con 0x564adbf4b170 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435009bed0 con 0x564adbf4b170 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f434c010430 con 0x564adbf4b170 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 10 client.16152 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 20 client.16152 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbf61260 0x564adbf63700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 4 client.16152 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564adbebda40 con 0x564adbf61260 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.829+0000 7f435b7fe640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbf61260 0x564adbf63700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.830+0000 7f435b7fe640 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbf61260 0x564adbf63700 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ae91df811c4792 server_cookie=571f2fba24719b60 in_seq=0 out_seq=0 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.830+0000 7f433b7fe640 10 client.16152 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:22.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f433b7fe640 1 -- 172.21.15.96:0/376547048 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564adbebda40 con 0x564adbf61260 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f433b7fe640 10 client.16152 handle_command_reply: tid=0 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 client.16152 shutdown 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 msgr2=0x564adbf55680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f434c012700 tx=0x7f434c012bc0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 shutdown_connections 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbf61260 0x564adbf63700 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 --2- 172.21.15.96:0/376547048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf55680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 >> 172.21.15.96:0/376547048 conn(0x564adbf466e0 msgr2=0x564adbf45900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f433b7fe640 0 client.16152 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:22.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 shutdown_connections 2024-03-18T15:42:22.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 1 -- 172.21.15.96:0/376547048 wait complete. 2024-03-18T15:42:22.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.831+0000 7f4364fc9740 20 client.16152 trim_cache size 0 max 16384 2024-03-18T15:42:22.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.848+0000 7f4364fc9740 1 Processor -- start 2024-03-18T15:42:22.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 -- start start 2024-03-18T15:42:22.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4b170 0x564adbf4a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf50cd0 0x564adbf69010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a8de0 con 0x564adbf4bb60 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1aaa40 con 0x564adbf50cd0 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4364fc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a94b0 con 0x564adbf4b170 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33568/0 (socket says 172.21.15.96:33568) 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 -- 172.21.15.96:0/4277780360 learned_addr learned my addr 172.21.15.96:0/4277780360 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f436097c640 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf50cd0 0x564adbf69010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435bfff640 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4b170 0x564adbf4a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 -- 172.21.15.96:0/4277780360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4b170 msgr2=0x564adbf4a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4b170 0x564adbf4a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 -- 172.21.15.96:0/4277780360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf50cd0 msgr2=0x564adbf69010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf50cd0 0x564adbf69010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 -- 172.21.15.96:0/4277780360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adbb840f0 con 0x564adbf4bb60 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f435b7fe640 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f435009b4d0 tx=0x7f43500d2940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7f6b11c49fec9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4341ffb640 1 -- 172.21.15.96:0/4277780360 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500ae070 con 0x564adbf4bb60 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4341ffb640 1 -- 172.21.15.96:0/4277780360 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435009bed0 con 0x564adbf4bb60 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.849+0000 7f4341ffb640 1 -- 172.21.15.96:0/4277780360 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500b3040 con 0x564adbf4bb60 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- 172.21.15.96:0/4277780360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 msgr2=0x564adbf5cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f435009b4d0 tx=0x7f43500d2940 comp rx=0 tx=0).stop 2024-03-18T15:42:22.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- 172.21.15.96:0/4277780360 shutdown_connections 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf50cd0 0x564adbf69010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4bb60 0x564adbf5cbc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 --2- 172.21.15.96:0/4277780360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf4b170 0x564adbf4a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- 172.21.15.96:0/4277780360 >> 172.21.15.96:0/4277780360 conn(0x564adbf466e0 msgr2=0x564adbf45070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- 172.21.15.96:0/4277780360 shutdown_connections 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- 172.21.15.96:0/4277780360 wait complete. 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.850+0000 7f4364fc9740 1 -- start start 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 Processor -- start 2024-03-18T15:42:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1aa2b0 con 0x564adbf4b170 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a9be0 con 0x564adbf4bb60 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a6dc0 con 0x564adbf50cd0 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f4364fc9740 10 client.0 _ll_register_callbacks cb 0x7f434c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33580/0 (socket says 172.21.15.96:33580) 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 -- 172.21.15.96:0/2458723712 learned_addr learned my addr 172.21.15.96:0/2458723712 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435b7fe640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f436097c640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 -- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 msgr2=0x564adbf5b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 -- 172.21.15.96:0/2458723712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 msgr2=0x564adbf5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 -- 172.21.15.96:0/2458723712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adbe779d0 con 0x564adbf4b170 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f435bfff640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f434c010a90 tx=0x7f434c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece96f26ef5edb6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.851+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c02b030 con 0x564adbf4b170 2024-03-18T15:42:22.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f434c018070 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f435009bed0 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f434c010b10 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 10 client.16158 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564adbfc76b0 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435009bed0 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f434c0035f0 con 0x564adbf4b170 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 10 client.16158 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 20 client.16158 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbfc8140 0x564adbfca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 4 client.16158 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564adbeb7d90 con 0x564adbfc8140 2024-03-18T15:42:22.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.852+0000 7f435b7fe640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbfc8140 0x564adbfca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f433b7fe640 10 client.16158 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f435b7fe640 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbfc8140 0x564adbfca560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1960b969d5126f2 server_cookie=5f1604cf40d4fa9d in_seq=0 out_seq=0 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f433b7fe640 1 -- 172.21.15.96:0/2458723712 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564adbeb7d90 con 0x564adbfc8140 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f433b7fe640 10 client.16158 handle_command_reply: tid=0 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 client.16158 shutdown 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 msgr2=0x564adbf5a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f434c010a90 tx=0x7f434c009f20 comp rx=0 tx=0).stop 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 shutdown_connections 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564adbf50cd0 0x564adbf5b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564adbf4bb60 0x564adbf5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564adbfc8140 0x564adbfca560 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 --2- 172.21.15.96:0/2458723712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564adbf4b170 0x564adbf5a470 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f433b7fe640 0 client.16158 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 >> 172.21.15.96:0/2458723712 conn(0x564adbf466e0 msgr2=0x564adbf4a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 shutdown_connections 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 1 -- 172.21.15.96:0/2458723712 wait complete. 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.853+0000 7f4364fc9740 20 client.16158 trim_cache size 0 max 16384 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:42:22.862 INFO:teuthology.orchestra.run.smithi096.stdout: "210876af-c1c5-487a-839c-dd82471180fb": { 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "210876af-c1c5-487a-839c-dd82471180fb", 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:42:22.863 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 msgr2=0x7f43440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 0x7f43440a4cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4354005e30 tx=0x7f4354032590 comp rx=0 tx=0).stop 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 msgr2=0x7f435c197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f435000e9f0 tx=0x7f435000edf0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 shutdown_connections 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f43440a2800 0x7f43440a4cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f435c1033a0 0x7f435c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1029e0 0x7f435c197ff0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 --2- 172.21.15.96:0/1479747246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f435c1017f0 0x7f435c197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:22.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 >> 172.21.15.96:0/1479747246 conn(0x7f435c0fcd80 msgr2=0x7f435c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:22.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.854+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 shutdown_connections 2024-03-18T15:42:22.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:22.855+0000 7f43623df640 1 -- 172.21.15.96:0/1479747246 wait complete. 2024-03-18T15:42:22.866 INFO:tasks.cephfs.filesystem:scrub status for tag:210876af-c1c5-487a-839c-dd82471180fb - {'path': '/', 'tag': '210876af-c1c5-487a-839c-dd82471180fb', 'options': 'recursive,force'} 2024-03-18T15:42:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:22 smithi193 ceph-mon[36702]: pgmap v420: 97 pgs: 97 active+clean; 225 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 431 op/s 2024-03-18T15:42:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:22 smithi096 ceph-mon[31930]: pgmap v420: 97 pgs: 97 active+clean; 225 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 431 op/s 2024-03-18T15:42:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:22 smithi137 ceph-mon[35868]: pgmap v420: 97 pgs: 97 active+clean; 225 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.5 MiB/s wr, 431 op/s 2024-03-18T15:42:24.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:24 smithi137 ceph-mon[35868]: pgmap v421: 97 pgs: 97 active+clean; 223 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 362 op/s 2024-03-18T15:42:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:24 smithi193 ceph-mon[36702]: pgmap v421: 97 pgs: 97 active+clean; 223 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 362 op/s 2024-03-18T15:42:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:24 smithi096 ceph-mon[31930]: pgmap v421: 97 pgs: 97 active+clean; 223 MiB data, 668 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 362 op/s 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 -- 172.21.15.96:0/3785229430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 msgr2=0x7fdf18102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 --2- 172.21.15.96:0/3785229430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18102ea0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009900 tx=0x7fdf0802bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 -- 172.21.15.96:0/3785229430 shutdown_connections 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 --2- 172.21.15.96:0/3785229430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 --2- 172.21.15.96:0/3785229430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18102ea0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 --2- 172.21.15.96:0/3785229430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf18101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 -- 172.21.15.96:0/3785229430 >> 172.21.15.96:0/3785229430 conn(0x7fdf180fcda0 msgr2=0x7fdf180ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 -- 172.21.15.96:0/3785229430 shutdown_connections 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 -- 172.21.15.96:0/3785229430 wait complete. 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.564+0000 7fdf20787640 1 Processor -- start 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 -- start start 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf181075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18107af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810d500 con 0x7fdf18102a20 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810d380 con 0x7fdf181033e0 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf20787640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1810d680 con 0x7fdf18101830 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf181075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:48468/0 (socket says 172.21.15.96:48468) 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48396/0 (socket says 172.21.15.96:48396) 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1081558755 learned_addr learned my addr 172.21.15.96:0/1081558755 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 -- 172.21.15.96:0/1081558755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 msgr2=0x7fdf18104120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 -- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 msgr2=0x7fdf18107af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18107af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 -- 172.21.15.96:0/1081558755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf181049e0 con 0x7fdf18101830 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.565+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf181075b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9c0 tx=0x7fdf1400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.566+0000 7fdf0f7fe640 1 -- 172.21.15.96:0/1081558755 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400ccc0 con 0x7fdf18101830 2024-03-18T15:42:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.566+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf181a60d0 con 0x7fdf18101830 2024-03-18T15:42:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.566+0000 7fdf0f7fe640 1 -- 172.21.15.96:0/1081558755 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf140035f0 con 0x7fdf18101830 2024-03-18T15:42:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.566+0000 7fdf0f7fe640 1 -- 172.21.15.96:0/1081558755 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400ce60 con 0x7fdf18101830 2024-03-18T15:42:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.566+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf181a64d0 con 0x7fdf18101830 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.568+0000 7fdf23371740 1 Processor -- start 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 -- start start 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4afb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 0x561bd4afe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a5ba40 con 0x561bd4af6d30 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a04920 con 0x561bd4afbea0 2024-03-18T15:42:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf23371740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a57ae0 con 0x561bd4af6340 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf0f7fe640 1 -- 172.21.15.96:0/1081558755 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fdf14010690 con 0x7fdf18101830 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42902/0 (socket says 172.21.15.96:42902) 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1782370487 learned_addr learned my addr 172.21.15.96:0/1782370487 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 0x561bd4afe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4afb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.569+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1782370487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 msgr2=0x561bd4af6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1782370487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 msgr2=0x561bd4afb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4afb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1782370487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd4a1c460 con 0x561bd4afbea0 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4afb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 0x561bd4afe330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf08036040 tx=0x7fdf0802c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4152fef75a776200 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf0f7fe640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 0x7fdf040a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf0cff9640 1 -- 172.21.15.96:0/1782370487 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08045070 con 0x561bd4afbea0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf0cff9640 1 -- 172.21.15.96:0/1782370487 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08004040 con 0x561bd4afbea0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf0cff9640 1 -- 172.21.15.96:0/1782370487 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004c90 con 0x561bd4afbea0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf0f7fe640 1 -- 172.21.15.96:0/1081558755 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdf140cef90 con 0x7fdf18101830 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.570+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 0x7fdf040a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- 172.21.15.96:0/1782370487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 msgr2=0x561bd4afe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 0x561bd4afe330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf08036040 tx=0x7fdf0802c680 comp rx=0 tx=0).stop 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 0x7fdf040a4d60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x561bd4af7590 tx=0x7fdf10009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- 172.21.15.96:0/1782370487 shutdown_connections 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4afbea0 0x561bd4afe330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4afb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 --2- 172.21.15.96:0/1782370487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6340 0x561bd4af6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- 172.21.15.96:0/1782370487 >> 172.21.15.96:0/1782370487 conn(0x561bd4af18b0 msgr2=0x561bd4af1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- 172.21.15.96:0/1782370487 shutdown_connections 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- 172.21.15.96:0/1782370487 wait complete. 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.571+0000 7fdf23371740 1 -- start start 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 Processor -- start 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 0x561bd4b00c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561bd4afedf0 con 0x561bd4afbea0 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561bd4a17d70 con 0x561bd4af6340 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561bd4afef70 con 0x561bd4af6d30 2024-03-18T15:42:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 0x561bd4b00c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48412/0 (socket says 172.21.15.96:48412) 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 0x561bd4b00c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42916/0 (socket says 172.21.15.96:42916) 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1169304989 learned_addr learned my addr 172.21.15.96:0/1169304989 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf23371740 10 client.0 _ll_register_callbacks cb 0x561bd476fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 msgr2=0x561bd4b00c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 0x561bd4b00c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1169304989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 msgr2=0x561bd4b006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1ed24640 1 -- 172.21.15.96:0/1169304989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd4770210 con 0x561bd4afbea0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.572+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf100122d0 tx=0x7fdf10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32db4500e7dc3b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.581 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10011d90 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdf10011020 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf08004040 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf10011220 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 10 client.16170 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd483c620 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf08004040 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fdf10027070 con 0x561bd4afbea0 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 10 client.16170 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 20 client.16170 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b0c600 0x561bd4b0ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 4 client.16170 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.573+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd4a68f80 con 0x561bd4b0c600 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.574+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b0c600 0x561bd4b0ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.574+0000 7fdf01ffb640 10 client.16170 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.574+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b0c600 0x561bd4b0ea90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529e2f1ed3db6e7 server_cookie=ee824d0d9f1bf530 in_seq=0 out_seq=0 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1169304989 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561bd4a68f80 con 0x561bd4b0c600 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf01ffb640 10 client.16170 handle_command_reply: tid=0 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 client.16170 shutdown 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 msgr2=0x561bd4b07be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fdf100122d0 tx=0x7fdf10004ab0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 shutdown_connections 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b0c600 0x561bd4b0ea90 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4afbea0 0x561bd4b07be0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4af6d30 0x561bd4b00c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf01ffb640 0 client.16170 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:25.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 --2- 172.21.15.96:0/1169304989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 >> 172.21.15.96:0/1169304989 conn(0x561bd4af18b0 msgr2=0x561bd4af0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 shutdown_connections 2024-03-18T15:42:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 1 -- 172.21.15.96:0/1169304989 wait complete. 2024-03-18T15:42:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.575+0000 7fdf23371740 20 client.16170 trim_cache size 0 max 16384 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 Processor -- start 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 -- start start 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4af4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4af5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181af2b0 con 0x561bd4af6d30 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181b0f10 con 0x561bd4af6340 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf23371740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181af980 con 0x561bd4afbea0 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1e523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4af4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48420/0 (socket says 172.21.15.96:48420) 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1ed24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4af5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 -- 172.21.15.96:0/2626491897 learned_addr learned my addr 172.21.15.96:0/2626491897 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 -- 172.21.15.96:0/2626491897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 msgr2=0x561bd4af5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4af5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 -- 172.21.15.96:0/2626491897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 msgr2=0x561bd4af4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4af4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1dd22640 1 -- 172.21.15.96:0/2626491897 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd4af75c0 con 0x561bd4af6d30 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.591+0000 7fdf1e523640 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4af4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.592+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdf08000c00 tx=0x7fdf08005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae435123d1fda059 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.592+0000 7fdf03fff640 1 -- 172.21.15.96:0/2626491897 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08045040 con 0x561bd4af6d30 2024-03-18T15:42:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.592+0000 7fdf03fff640 1 -- 172.21.15.96:0/2626491897 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf080046c0 con 0x561bd4af6d30 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.592+0000 7fdf03fff640 1 -- 172.21.15.96:0/2626491897 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08005110 con 0x561bd4af6d30 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- 172.21.15.96:0/2626491897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 msgr2=0x561bd4b07ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdf08000c00 tx=0x7fdf08005b50 comp rx=0 tx=0).stop 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- 172.21.15.96:0/2626491897 shutdown_connections 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4af5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b07ba0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- 172.21.15.96:0/2626491897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4af4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- 172.21.15.96:0/2626491897 >> 172.21.15.96:0/2626491897 conn(0x561bd4af18b0 msgr2=0x561bd4af8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- 172.21.15.96:0/2626491897 shutdown_connections 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- 172.21.15.96:0/2626491897 wait complete. 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- start start 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 Processor -- start 2024-03-18T15:42:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b05200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4b06700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181b0780 con 0x561bd4af6d30 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181b00b0 con 0x561bd4af6340 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdf181ad290 con 0x561bd4afbea0 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf23371740 10 client.0 _ll_register_callbacks cb 0x7fdf10012190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.593+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1e523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b05200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48430/0 (socket says 172.21.15.96:48430) 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1688119222 learned_addr learned my addr 172.21.15.96:0/1688119222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4b06700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 msgr2=0x561bd4b06700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4b06700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1688119222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 msgr2=0x561bd4b05200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b05200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 -- 172.21.15.96:0/1688119222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd4b06cf0 con 0x561bd4af6d30 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b05200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1ed24640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4b06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf1dd22640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ce30 tx=0x7fdf08009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b9fa3b3ed1df4f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08058060 con 0x561bd4af6d30 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdf080050c0 con 0x561bd4af6d30 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf080046c0 con 0x561bd4af6d30 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf08045070 con 0x561bd4af6d30 2024-03-18T15:42:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf23371740 10 client.16176 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.594+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bd4b729d0 con 0x561bd4af6d30 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf080046c0 con 0x561bd4af6d30 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fdf0804ab00 con 0x561bd4af6d30 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 10 client.16176 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 20 client.16176 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b72ee0 0x561bd4b752d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 4 client.16176 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd4a632d0 con 0x561bd4b72ee0 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b72ee0 0x561bd4b752d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf01ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.595+0000 7fdf1e523640 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b72ee0 0x561bd4b752d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d52aad1bf4eec93d server_cookie=25091347597d90cf in_seq=0 out_seq=0 2024-03-18T15:42:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf01ffb640 1 -- 172.21.15.96:0/1688119222 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bd4a632d0 con 0x561bd4b72ee0 2024-03-18T15:42:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf01ffb640 10 client.16176 handle_command_reply: tid=0 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 client.16176 shutdown 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 msgr2=0x561bd4b05740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdf0802ce30 tx=0x7fdf08009660 comp rx=0 tx=0).stop 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 shutdown_connections 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561bd4afbea0 0x561bd4b06700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561bd4b72ee0 0x561bd4b752d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561bd4af6d30 0x561bd4b05740 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf01ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 --2- 172.21.15.96:0/1688119222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561bd4af6340 0x561bd4b05200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 >> 172.21.15.96:0/1688119222 conn(0x561bd4af18b0 msgr2=0x561bd4af4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 shutdown_connections 2024-03-18T15:42:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 1 -- 172.21.15.96:0/1688119222 wait complete. 2024-03-18T15:42:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.603+0000 7fdf23371740 20 client.16176 trim_cache size 0 max 16384 2024-03-18T15:42:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:42:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:42:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:42:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:42:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 msgr2=0x7fdf040a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 0x7fdf040a4d60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x561bd4af7590 tx=0x7fdf10009500 comp rx=0 tx=0).stop 2024-03-18T15:42:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 msgr2=0x7fdf181075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf181075b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400e9c0 tx=0x7fdf1400ee80 comp rx=0 tx=0).stop 2024-03-18T15:42:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 shutdown_connections 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdf040a28a0 0x7fdf040a4d60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf181033e0 0x7fdf18104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf18102a20 0x7fdf18107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 --2- 172.21.15.96:0/1081558755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf18101830 0x7fdf181075b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 >> 172.21.15.96:0/1081558755 conn(0x7fdf180fcda0 msgr2=0x7fdf18073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 shutdown_connections 2024-03-18T15:42:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:25.604+0000 7fdf20787640 1 -- 172.21.15.96:0/1081558755 wait complete. 2024-03-18T15:42:26.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:26 smithi193 ceph-mon[36702]: pgmap v422: 97 pgs: 97 active+clean; 230 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-03-18T15:42:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:26 smithi096 ceph-mon[31930]: pgmap v422: 97 pgs: 97 active+clean; 230 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-03-18T15:42:27.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:26 smithi137 ceph-mon[35868]: pgmap v422: 97 pgs: 97 active+clean; 230 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-03-18T15:42:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:28 smithi193 ceph-mon[36702]: pgmap v423: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-03-18T15:42:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:28 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:42:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:28 smithi096 ceph-mon[31930]: pgmap v423: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-03-18T15:42:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:28 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:42:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:28 smithi137 ceph-mon[35868]: pgmap v423: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-03-18T15:42:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:28 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:42:30.605 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: pgmap v424: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:42:30.606 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: pgmap v424: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:42:30.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:42:30.669 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:42:30.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: pgmap v424: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 142 op/s 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:42:30.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:42:31.328 INFO:tasks.workunit.client.0.smithi096.stdout: 56 1284898 459 980503 275 734093 1145 2024-03-18T15:42:32.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:32 smithi137 ceph-mon[35868]: pgmap v425: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1019 KiB/s wr, 119 op/s 2024-03-18T15:42:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:32 smithi193 ceph-mon[36702]: pgmap v425: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1019 KiB/s wr, 119 op/s 2024-03-18T15:42:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:32 smithi096 ceph-mon[31930]: pgmap v425: 97 pgs: 97 active+clean; 230 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1019 KiB/s wr, 119 op/s 2024-03-18T15:42:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:34 smithi137 ceph-mon[35868]: pgmap v426: 97 pgs: 97 active+clean; 236 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-18T15:42:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:34 smithi193 ceph-mon[36702]: pgmap v426: 97 pgs: 97 active+clean; 236 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-18T15:42:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:34 smithi096 ceph-mon[31930]: pgmap v426: 97 pgs: 97 active+clean; 236 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-03-18T15:42:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:36 smithi137 ceph-mon[35868]: pgmap v427: 97 pgs: 97 active+clean; 253 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-03-18T15:42:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:36 smithi193 ceph-mon[36702]: pgmap v427: 97 pgs: 97 active+clean; 253 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-03-18T15:42:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:36 smithi096 ceph-mon[31930]: pgmap v427: 97 pgs: 97 active+clean; 253 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-03-18T15:42:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 -- 172.21.15.96:0/1020187725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d41022d0 msgr2=0x7f89d4108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 --2- 172.21.15.96:0/1020187725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d41022d0 0x7f89d4108b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 -- 172.21.15.96:0/1020187725 shutdown_connections 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 --2- 172.21.15.96:0/1020187725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d41022d0 0x7f89d4108b60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 --2- 172.21.15.96:0/1020187725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4101910 0x7f89d4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 --2- 172.21.15.96:0/1020187725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4100720 0x7f89d4100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 -- 172.21.15.96:0/1020187725 >> 172.21.15.96:0/1020187725 conn(0x7f89d40fbcb0 msgr2=0x7f89d40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 -- 172.21.15.96:0/1020187725 shutdown_connections 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.697+0000 7f89da0dc640 1 -- 172.21.15.96:0/1020187725 wait complete. 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 Processor -- start 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 -- start start 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4100720 0x7f89d41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 0x7f89d419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410c3a0 con 0x7f89d4101910 2024-03-18T15:42:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410c220 con 0x7f89d41022d0 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89da0dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89d410c520 con 0x7f89d4100720 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53584/0 (socket says 172.21.15.96:53584) 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 -- 172.21.15.96:0/4063978872 learned_addr learned my addr 172.21.15.96:0/4063978872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d3fff640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 0x7f89d419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d37fe640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4100720 0x7f89d41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 -- 172.21.15.96:0/4063978872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4100720 msgr2=0x7f89d41979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4100720 0x7f89d41979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 -- 172.21.15.96:0/4063978872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 msgr2=0x7f89d419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 0x7f89d419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d2ffd640 1 -- 172.21.15.96:0/4063978872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d419f610 con 0x7f89d4101910 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.698+0000 7f89d3fff640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 0x7f89d419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89d0ff9640 1 -- 172.21.15.96:0/4063978872 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800cd80 con 0x7f89d4101910 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d419f900 con 0x7f89d4101910 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89d0ff9640 1 -- 172.21.15.96:0/4063978872 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8005500 con 0x7f89d4101910 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89d0ff9640 1 -- 172.21.15.96:0/4063978872 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010690 con 0x7f89d4101910 2024-03-18T15:42:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.699+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d419fe50 con 0x7f89d4101910 2024-03-18T15:42:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.701+0000 7f89d0ff9640 1 -- 172.21.15.96:0/4063978872 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f89c80046c0 con 0x7f89d4101910 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.701+0000 7f89dcda2740 1 Processor -- start 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 -- start start 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2ac8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 0x5650e2acd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650e2a2da40 con 0x5650e2acdea0 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650e29d6920 con 0x5650e2ac8d30 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89dcda2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650e2a29ae0 con 0x5650e2ac8340 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53588/0 (socket says 172.21.15.96:53588) 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 -- 172.21.15.96:0/3164856275 learned_addr learned my addr 172.21.15.96:0/3164856275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d3fff640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2ac8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d37fe640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 0x5650e2acd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d0ff9640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 0x7f89bc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d3fff640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 0x7f89bc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 -- 172.21.15.96:0/3164856275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 msgr2=0x5650e2ac8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2ac8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 -- 172.21.15.96:0/3164856275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 msgr2=0x5650e2acd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 0x5650e2acd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 -- 172.21.15.96:0/3164856275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e29ee460 con 0x5650e2acdea0 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d37fe640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 0x5650e2acd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d0ff9640 1 -- 172.21.15.96:0/4063978872 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f89c80cee70 con 0x7f89d4101910 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.702+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b4e0 tx=0x7f89c809b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a920a05f1581b322 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89d3fff640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 0x7f89bc0a4d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005e30 tx=0x7f89cc02c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89b9ffb640 1 -- 172.21.15.96:0/3164856275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c809b320 con 0x5650e2acdea0 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89b9ffb640 1 -- 172.21.15.96:0/3164856275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c809bee0 con 0x5650e2acdea0 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89b9ffb640 1 -- 172.21.15.96:0/3164856275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c809b140 con 0x5650e2acdea0 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 -- 172.21.15.96:0/3164856275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 msgr2=0x5650e2ad0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89c809b4e0 tx=0x7f89c809b6e0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 -- 172.21.15.96:0/3164856275 shutdown_connections 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad0330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8d30 0x5650e2acd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 --2- 172.21.15.96:0/3164856275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2ac8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 -- 172.21.15.96:0/3164856275 >> 172.21.15.96:0/3164856275 conn(0x5650e2ac38b0 msgr2=0x5650e2ac3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 -- 172.21.15.96:0/3164856275 shutdown_connections 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.703+0000 7f89dcda2740 1 -- 172.21.15.96:0/3164856275 wait complete. 2024-03-18T15:42:38.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 -- start start 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 Processor -- start 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ad25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ad2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650e2ad0df0 con 0x5650e2acdea0 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650e29e9d70 con 0x5650e2ac8340 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650e2ad0f70 con 0x5650e2ac8d30 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89dcda2740 10 client.0 _ll_register_callbacks cb 0x5650e2741a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ad25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53592/0 (socket says 172.21.15.96:53592) 2024-03-18T15:42:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 -- 172.21.15.96:0/82179588 learned_addr learned my addr 172.21.15.96:0/82179588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ad2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 -- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 msgr2=0x5650e2ad2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ad2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 -- 172.21.15.96:0/82179588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 msgr2=0x5650e2ad25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ad25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d3fff640 1 -- 172.21.15.96:0/82179588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e2742210 con 0x5650e2acdea0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.704+0000 7f89d37fe640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ad25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89d3fff640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287bebbb9d73c386 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc053060 con 0x5650e2acdea0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f89cc0349f0 con 0x5650e2acdea0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89c809bee0 con 0x5650e2acdea0 2024-03-18T15:42:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89cc047070 con 0x5650e2acdea0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650e280e620 con 0x5650e2acdea0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c809bee0 con 0x5650e2acdea0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f89cc03f070 con 0x5650e2acdea0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 10 client.16188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 20 client.16188 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2ade4f0 0x5650e2ae0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 4 client.16188 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e2a3af80 con 0x5650e2ade4f0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.705+0000 7f89d37fe640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2ade4f0 0x5650e2ae0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.706+0000 7f89b2ffd640 10 client.16188 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.706+0000 7f89d37fe640 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2ade4f0 0x5650e2ae0950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc02bc6c5f6ffdc server_cookie=6219ab777e2b63ea in_seq=0 out_seq=0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89b2ffd640 1 -- 172.21.15.96:0/82179588 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5650e2a3af80 con 0x5650e2ade4f0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89b2ffd640 10 client.16188 handle_command_reply: tid=0 2024-03-18T15:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 client.16188 shutdown 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 msgr2=0x5650e2ad9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc0095c0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 shutdown_connections 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2ade4f0 0x5650e2ae0950 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ad9b00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ad2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89b2ffd640 0 client.16188 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 --2- 172.21.15.96:0/82179588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ad25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 >> 172.21.15.96:0/82179588 conn(0x5650e2ac38b0 msgr2=0x5650e2ac2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 shutdown_connections 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 1 -- 172.21.15.96:0/82179588 wait complete. 2024-03-18T15:42:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.707+0000 7f89dcda2740 20 client.16188 trim_cache size 0 max 16384 2024-03-18T15:42:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 Processor -- start 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 -- start start 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ac3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ac6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41a8ce0 con 0x5650e2acdea0 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41aa940 con 0x5650e2ac8340 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.723+0000 7f89dcda2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41a93b0 con 0x5650e2ac8d30 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53604/0 (socket says 172.21.15.96:53604) 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 -- 172.21.15.96:0/1148824625 learned_addr learned my addr 172.21.15.96:0/1148824625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d37fe640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ac3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ac6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 -- 172.21.15.96:0/1148824625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 msgr2=0x5650e2ac6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ac6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 -- 172.21.15.96:0/1148824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 msgr2=0x5650e2ac3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ac3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 -- 172.21.15.96:0/1148824625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e2ae0fd0 con 0x5650e2acdea0 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ac6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89d3fff640 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f89cc032fd0 tx=0x7f89cc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d6f6d0067332ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89b97fa640 1 -- 172.21.15.96:0/1148824625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc053070 con 0x5650e2acdea0 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89b97fa640 1 -- 172.21.15.96:0/1148824625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89cc0347f0 con 0x5650e2acdea0 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.724+0000 7f89b97fa640 1 -- 172.21.15.96:0/1148824625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc039040 con 0x5650e2acdea0 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- 172.21.15.96:0/1148824625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 msgr2=0x5650e2ac7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f89cc032fd0 tx=0x7f89cc005fa0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- 172.21.15.96:0/1148824625 shutdown_connections 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2acdea0 0x5650e2ac7050 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8d30 0x5650e2ac6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1148824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2ac8340 0x5650e2ac3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- 172.21.15.96:0/1148824625 >> 172.21.15.96:0/1148824625 conn(0x5650e2ac38b0 msgr2=0x5650e2abea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- 172.21.15.96:0/1148824625 shutdown_connections 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- 172.21.15.96:0/1148824625 wait complete. 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 -- start start 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.725+0000 7f89dcda2740 1 Processor -- start 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2b3fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2acdea0 0x5650e2b472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41aa1b0 con 0x5650e2ac8d30 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41a9ae0 con 0x5650e2acdea0 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89d41a6cc0 con 0x5650e2ac8340 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 10 client.0 _ll_register_callbacks cb 0x7f89c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:38.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53620/0 (socket says 172.21.15.96:53620) 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 -- 172.21.15.96:0/1431913751 learned_addr learned my addr 172.21.15.96:0/1431913751 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d37fe640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2b3fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d3fff640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2acdea0 0x5650e2b472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 -- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 msgr2=0x5650e2b3fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2b3fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 -- 172.21.15.96:0/1431913751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2acdea0 msgr2=0x5650e2b472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2acdea0 0x5650e2b472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 -- 172.21.15.96:0/1431913751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e2b478a0 con 0x5650e2ac8d30 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89d2ffd640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89c80981b0 tx=0x7f89c809f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f85b691a917027 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80b3070 con 0x5650e2ac8d30 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f89c800bda0 con 0x5650e2ac8d30 2024-03-18T15:42:38.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.726+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89cc0347f0 con 0x5650e2ac8d30 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89c809b140 con 0x5650e2ac8d30 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650e2b48c50 con 0x5650e2ac8d30 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89cc0347f0 con 0x5650e2ac8d30 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f89c80860b0 con 0x5650e2ac8d30 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 20 client.16194 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2b492c0 0x5650e2b4b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 4 client.16194 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650e2a352d0 con 0x5650e2b492c0 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.727+0000 7f89d37fe640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2b492c0 0x5650e2b4b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.728+0000 7f89d37fe640 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2b492c0 0x5650e2b4b6e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20395bd2aacb99a server_cookie=c54e582b9fc719f3 in_seq=0 out_seq=0 2024-03-18T15:42:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.728+0000 7f89b2ffd640 10 client.16194 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89b2ffd640 1 -- 172.21.15.96:0/1431913751 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650e2a352d0 con 0x5650e2b492c0 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89b2ffd640 10 client.16194 handle_command_reply: tid=0 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 client.16194 shutdown 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 msgr2=0x5650e2b40270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89c80981b0 tx=0x7f89c809f930 comp rx=0 tx=0).stop 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 shutdown_connections 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650e2acdea0 0x5650e2b472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5650e2b492c0 0x5650e2b4b6e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650e2ac8d30 0x5650e2b40270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 --2- 172.21.15.96:0/1431913751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650e2ac8340 0x5650e2b3fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89b2ffd640 0 client.16194 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.735+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 >> 172.21.15.96:0/1431913751 conn(0x5650e2ac38b0 msgr2=0x5650e2ac3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 shutdown_connections 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89dcda2740 1 -- 172.21.15.96:0/1431913751 wait complete. 2024-03-18T15:42:38.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89dcda2740 20 client.16194 trim_cache size 0 max 16384 2024-03-18T15:42:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:42:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:42:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:42:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 msgr2=0x7f89bc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 0x7f89bc0a4d60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f89cc005e30 tx=0x7f89cc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 msgr2=0x7f89d4197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800edf0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 shutdown_connections 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f89bc0a28a0 0x7f89bc0a4d60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d41022d0 0x7f89d419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89d4101910 0x7f89d4197f30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 --2- 172.21.15.96:0/4063978872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89d4100720 0x7f89d41979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 >> 172.21.15.96:0/4063978872 conn(0x7f89d40fbcb0 msgr2=0x7f89d40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 shutdown_connections 2024-03-18T15:42:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:38.736+0000 7f89da0dc640 1 -- 172.21.15.96:0/4063978872 wait complete. 2024-03-18T15:42:38.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:38 smithi137 ceph-mon[35868]: pgmap v428: 97 pgs: 97 active+clean; 268 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-03-18T15:42:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:38 smithi193 ceph-mon[36702]: pgmap v428: 97 pgs: 97 active+clean; 268 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-03-18T15:42:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:38 smithi096 ceph-mon[31930]: pgmap v428: 97 pgs: 97 active+clean; 268 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-03-18T15:42:40.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:40 smithi137 ceph-mon[35868]: pgmap v429: 97 pgs: 97 active+clean; 297 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 443 op/s 2024-03-18T15:42:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:40 smithi193 ceph-mon[36702]: pgmap v429: 97 pgs: 97 active+clean; 297 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 443 op/s 2024-03-18T15:42:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:40 smithi096 ceph-mon[31930]: pgmap v429: 97 pgs: 97 active+clean; 297 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 443 op/s 2024-03-18T15:42:41.328 INFO:tasks.workunit.client.0.smithi096.stdout: 59 590856 166 442257 175 328457 587 2024-03-18T15:42:42.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:42 smithi137 ceph-mon[35868]: pgmap v430: 97 pgs: 97 active+clean; 296 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 439 op/s 2024-03-18T15:42:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:42 smithi193 ceph-mon[36702]: pgmap v430: 97 pgs: 97 active+clean; 296 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 439 op/s 2024-03-18T15:42:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:42 smithi096 ceph-mon[31930]: pgmap v430: 97 pgs: 97 active+clean; 296 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.2 MiB/s wr, 439 op/s 2024-03-18T15:42:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:44 smithi137 ceph-mon[35868]: pgmap v431: 97 pgs: 97 active+clean; 295 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:42:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:44 smithi193 ceph-mon[36702]: pgmap v431: 97 pgs: 97 active+clean; 295 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:42:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:44 smithi096 ceph-mon[31930]: pgmap v431: 97 pgs: 97 active+clean; 295 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:42:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:46 smithi137 ceph-mon[35868]: pgmap v432: 97 pgs: 97 active+clean; 301 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-03-18T15:42:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:46 smithi193 ceph-mon[36702]: pgmap v432: 97 pgs: 97 active+clean; 301 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-03-18T15:42:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:46 smithi096 ceph-mon[31930]: pgmap v432: 97 pgs: 97 active+clean; 301 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 476 op/s 2024-03-18T15:42:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:48 smithi137 ceph-mon[35868]: pgmap v433: 97 pgs: 97 active+clean; 300 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-03-18T15:42:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:48 smithi193 ceph-mon[36702]: pgmap v433: 97 pgs: 97 active+clean; 300 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-03-18T15:42:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:48 smithi096 ceph-mon[31930]: pgmap v433: 97 pgs: 97 active+clean; 300 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 368 op/s 2024-03-18T15:42:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:50 smithi137 ceph-mon[35868]: pgmap v434: 97 pgs: 97 active+clean; 305 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-03-18T15:42:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:50 smithi193 ceph-mon[36702]: pgmap v434: 97 pgs: 97 active+clean; 305 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-03-18T15:42:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:50 smithi096 ceph-mon[31930]: pgmap v434: 97 pgs: 97 active+clean; 305 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-03-18T15:42:51.328 INFO:tasks.workunit.client.0.smithi096.stdout: 67 1476761 383 1120897 354 825550 1036 2024-03-18T15:42:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 -- 172.21.15.96:0/2538586908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0100fd0 msgr2=0x7efcc0101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2538586908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0100fd0 0x7efcc0101450 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009900 tx=0x7efcb402bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 -- 172.21.15.96:0/2538586908 shutdown_connections 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2538586908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc0101990 0x7efcc01082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2538586908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0100fd0 0x7efcc0101450 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2538586908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc01002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.834+0000 7efcc59b2640 1 -- 172.21.15.96:0/2538586908 >> 172.21.15.96:0/2538586908 conn(0x7efcc00fb430 msgr2=0x7efcc00fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- 172.21.15.96:0/2538586908 shutdown_connections 2024-03-18T15:42:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- 172.21.15.96:0/2538586908 wait complete. 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 Processor -- start 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- start start 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc00fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc019cf80 0x7efcc019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010bb10 con 0x7efcc0101990 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010b990 con 0x7efcc00ffe80 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.835+0000 7efcc59b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcc010bc90 con 0x7efcc019cf80 2024-03-18T15:42:51.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc00fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50990/0 (socket says 172.21.15.96:50990) 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbe7fc640 1 -- 172.21.15.96:0/2208091080 learned_addr learned my addr 172.21.15.96:0/2208091080 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbf7fe640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc019cf80 0x7efcc019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 -- 172.21.15.96:0/2208091080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc019cf80 msgr2=0x7efcc019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc019cf80 0x7efcc019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 -- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 msgr2=0x7efcc019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 -- 172.21.15.96:0/2208091080 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc019fd00 con 0x7efcc00ffe80 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc00fd680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.836+0000 7efcc49b0640 1 -- 172.21.15.96:0/2208091080 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000ccc0 con 0x7efcc00ffe80 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.837+0000 7efcc49b0640 1 -- 172.21.15.96:0/2208091080 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0010910 con 0x7efcc00ffe80 2024-03-18T15:42:51.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.837+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc019fff0 con 0x7efcc00ffe80 2024-03-18T15:42:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.837+0000 7efcc49b0640 1 -- 172.21.15.96:0/2208091080 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0018960 con 0x7efcc00ffe80 2024-03-18T15:42:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.837+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01967c0 con 0x7efcc00ffe80 2024-03-18T15:42:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc49b0640 1 -- 172.21.15.96:0/2208091080 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efcb0010ac0 con 0x7efcc00ffe80 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc49b0640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 0x7efca80a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc859e740 1 Processor -- start 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc859e740 1 -- start start 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc49b0640 1 -- 172.21.15.96:0/2208091080 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efcb00cee40 con 0x7efcc00ffe80 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 0x7efca80a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc859e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.839+0000 7efcc859e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890d30 0x560bef895830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcc859e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef898330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcc859e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560bef7f5a40 con 0x560bef890340 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcc859e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560bef79e920 con 0x560bef890d30 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcc859e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560bef7f1ae0 con 0x560bef895ea0 2024-03-18T15:42:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50998/0 (socket says 172.21.15.96:50998) 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 -- 172.21.15.96:0/2143826528 learned_addr learned my addr 172.21.15.96:0/2143826528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbf7fe640 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef898330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890d30 0x560bef895830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 -- 172.21.15.96:0/2143826528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 msgr2=0x560bef898330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef898330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 -- 172.21.15.96:0/2143826528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890d30 msgr2=0x560bef895830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890d30 0x560bef895830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 -- 172.21.15.96:0/2143826528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bef7b6460 con 0x560bef890340 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 0x7efca80a9170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efcb4005ab0 tx=0x7efcb40041c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efcb00a3a20 tx=0x7efcb009b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d5d483a9a89fe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efca5ffb640 1 -- 172.21.15.96:0/2143826528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0099900 con 0x560bef890340 2024-03-18T15:42:51.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efca5ffb640 1 -- 172.21.15.96:0/2143826528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0099b60 con 0x560bef890340 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.840+0000 7efca5ffb640 1 -- 172.21.15.96:0/2143826528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb009af10 con 0x560bef890340 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 -- 172.21.15.96:0/2143826528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 msgr2=0x560bef890760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efcb00a3a20 tx=0x7efcb009b740 comp rx=0 tx=0).stop 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 -- 172.21.15.96:0/2143826528 shutdown_connections 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef898330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890d30 0x560bef895830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 --2- 172.21.15.96:0/2143826528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef890760 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 -- 172.21.15.96:0/2143826528 >> 172.21.15.96:0/2143826528 conn(0x560bef88b8b0 msgr2=0x560bef88bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 -- 172.21.15.96:0/2143826528 shutdown_connections 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.841+0000 7efcc859e740 1 -- 172.21.15.96:0/2143826528 wait complete. 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 -- start start 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 Processor -- start 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef89ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef8a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560bef898df0 con 0x560bef890d30 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560bef7b1d70 con 0x560bef890340 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560bef898f70 con 0x560bef895ea0 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcc859e740 10 client.0 _ll_register_callbacks cb 0x560bef509a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.842+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef89ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55252/0 (socket says 172.21.15.96:55252) 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbeffd640 1 -- 172.21.15.96:0/2379499673 learned_addr learned my addr 172.21.15.96:0/2379499673 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbf7fe640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef8a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 -- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 msgr2=0x560bef8a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef8a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 -- 172.21.15.96:0/2379499673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 msgr2=0x560bef89a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 -- 172.21.15.96:0/2379499673 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bef50a210 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef89ac20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efcb4000c00 tx=0x7efcb402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d78e7a8637e9017 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4051070 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efcb40055e0 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb0099b60 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcb405d050 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.843+0000 7efcc859e740 10 client.16206 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.844+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bef5d6620 con 0x560bef890d30 2024-03-18T15:42:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.844+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0099b60 con 0x560bef890d30 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7efcb40035f0 con 0x560bef890d30 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 10 client.16206 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 20 client.16206 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef8a65b0 0x560bef8a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 4 client.16206 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bef802f80 con 0x560bef8a65b0 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef8a65b0 0x560bef8a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efcbeffd640 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef8a65b0 0x560bef8a89c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4029c09fea43d5 server_cookie=726ab9b0afe0bc20 in_seq=0 out_seq=0 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.845+0000 7efc9effd640 10 client.16206 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efc9effd640 1 -- 172.21.15.96:0/2379499673 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560bef802f80 con 0x560bef8a65b0 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efc9effd640 10 client.16206 handle_command_reply: tid=0 2024-03-18T15:42:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efcc859e740 1 client.16206 shutdown 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 msgr2=0x560bef89ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef89ac20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efcb4000c00 tx=0x7efcb402bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.846+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 shutdown_connections 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef895ea0 0x560bef8a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef8a65b0 0x560bef8a89c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef89ac20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 --2- 172.21.15.96:0/2379499673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef890340 0x560bef89a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efc9effd640 0 client.16206 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 >> 172.21.15.96:0/2379499673 conn(0x560bef88b8b0 msgr2=0x560bef88abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 shutdown_connections 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 1 -- 172.21.15.96:0/2379499673 wait complete. 2024-03-18T15:42:51.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.847+0000 7efcc859e740 20 client.16206 trim_cache size 0 max 16384 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 Processor -- start 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 -- start start 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890340 0x560bef88eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef88f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.861+0000 7efcc859e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a70b0 con 0x560bef890d30 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcc859e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a8d10 con 0x560bef895ea0 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcc859e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a7780 con 0x560bef890340 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51018/0 (socket says 172.21.15.96:51018) 2024-03-18T15:42:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 -- 172.21.15.96:0/3845936955 learned_addr learned my addr 172.21.15.96:0/3845936955 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbeffd640 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890340 0x560bef88eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbf7fe640 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef88f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 -- 172.21.15.96:0/3845936955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890340 msgr2=0x560bef88eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890340 0x560bef88eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 -- 172.21.15.96:0/3845936955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 msgr2=0x560bef88f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef88f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 -- 172.21.15.96:0/3845936955 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bef8953f0 con 0x560bef890d30 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efcb4004ec0 tx=0x7efcb402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24ecb8a7d29b31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efca57fa640 1 -- 172.21.15.96:0/3845936955 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4051070 con 0x560bef890d30 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efca57fa640 1 -- 172.21.15.96:0/3845936955 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb4009520 con 0x560bef890d30 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.862+0000 7efca57fa640 1 -- 172.21.15.96:0/3845936955 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb404d6b0 con 0x560bef890d30 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- 172.21.15.96:0/3845936955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 msgr2=0x560bef8a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7efcb4004ec0 tx=0x7efcb402c330 comp rx=0 tx=0).stop 2024-03-18T15:42:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- 172.21.15.96:0/3845936955 shutdown_connections 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef88f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890d30 0x560bef8a1bb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 --2- 172.21.15.96:0/3845936955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890340 0x560bef88eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- 172.21.15.96:0/3845936955 >> 172.21.15.96:0/3845936955 conn(0x560bef88b8b0 msgr2=0x560bef89a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- 172.21.15.96:0/3845936955 shutdown_connections 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- 172.21.15.96:0/3845936955 wait complete. 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 -- start start 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.863+0000 7efcc859e740 1 Processor -- start 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 0x560bef89f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef8a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a8580 con 0x560bef890340 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a7eb0 con 0x560bef895ea0 2024-03-18T15:42:51.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01a5090 con 0x560bef890d30 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 10 client.0 _ll_register_callbacks cb 0x7efcb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51022/0 (socket says 172.21.15.96:51022) 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 -- 172.21.15.96:0/1579093588 learned_addr learned my addr 172.21.15.96:0/1579093588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbf7fe640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef8a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 0x560bef89f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 -- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 msgr2=0x560bef89f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 0x560bef89f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 -- 172.21.15.96:0/1579093588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 msgr2=0x560bef8a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef8a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 -- 172.21.15.96:0/1579093588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bef8a0fa0 con 0x560bef890340 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 0x560bef89f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:51.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcbeffd640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efcb00cf160 tx=0x7efcb0097c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51740671eb10fe6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00afb90 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efcb00afd30 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb4009520 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efcb009acb0 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.864+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bef90ca80 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb4009520 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7efcb0018b90 con 0x560bef890340 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 20 client.16212 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef90d100 0x560bef90f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 4 client.16212 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:42:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bef7fd2d0 con 0x560bef90d100 2024-03-18T15:42:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef90d100 0x560bef90f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efc9effd640 10 client.16212 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.865+0000 7efcbe7fc640 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef90d100 0x560bef90f4f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689ce24705257113 server_cookie=a6a4e7138dec7e43 in_seq=0 out_seq=0 2024-03-18T15:42:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efc9effd640 1 -- 172.21.15.96:0/1579093588 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bef7fd2d0 con 0x560bef90d100 2024-03-18T15:42:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efc9effd640 10 client.16212 handle_command_reply: tid=0 2024-03-18T15:42:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 client.16212 shutdown 2024-03-18T15:42:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 msgr2=0x560bef89f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efcb00cf160 tx=0x7efcb0097c30 comp rx=0 tx=0).stop 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 shutdown_connections 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560bef895ea0 0x560bef8a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560bef890d30 0x560bef89f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560bef90d100 0x560bef90f4f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 --2- 172.21.15.96:0/1579093588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560bef890340 0x560bef89f460 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efc9effd640 0 client.16212 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 >> 172.21.15.96:0/1579093588 conn(0x560bef88b8b0 msgr2=0x560bef88eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 shutdown_connections 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 1 -- 172.21.15.96:0/1579093588 wait complete. 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.873+0000 7efcc859e740 20 client.16212 trim_cache size 0 max 16384 2024-03-18T15:42:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:42:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:42:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:42:51.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 msgr2=0x7efca80a9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 0x7efca80a9170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7efcb4005ab0 tx=0x7efcb40041c0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 msgr2=0x7efcc00fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc00fd680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcb000e9f0 tx=0x7efcb000eeb0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 shutdown_connections 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efca80a6cb0 0x7efca80a9170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcc019cf80 0x7efcc019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcc0101990 0x7efcc019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 --2- 172.21.15.96:0/2208091080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcc00ffe80 0x7efcc00fd680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 >> 172.21.15.96:0/2208091080 conn(0x7efcc00fb430 msgr2=0x7efcc0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 shutdown_connections 2024-03-18T15:42:51.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:42:51.874+0000 7efcc59b2640 1 -- 172.21.15.96:0/2208091080 wait complete. 2024-03-18T15:42:52.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:52 smithi137 ceph-mon[35868]: pgmap v435: 97 pgs: 97 active+clean; 304 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T15:42:52.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:42:52.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:42:52.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:52 smithi096 ceph-mon[31930]: pgmap v435: 97 pgs: 97 active+clean; 304 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T15:42:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:52 smithi193 ceph-mon[36702]: pgmap v435: 97 pgs: 97 active+clean; 304 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-03-18T15:42:52.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 -- 172.21.15.96:0/93502389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1018d0 msgr2=0x7f19fc101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 --2- 172.21.15.96:0/93502389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1018d0 0x7f19fc101d50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009900 tx=0x7f19ec02bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 -- 172.21.15.96:0/93502389 shutdown_connections 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 --2- 172.21.15.96:0/93502389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 0x7f19fc108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 --2- 172.21.15.96:0/93502389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1018d0 0x7f19fc101d50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 --2- 172.21.15.96:0/93502389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1006e0 0x7f19fc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.983+0000 7f1a033f4640 1 -- 172.21.15.96:0/93502389 >> 172.21.15.96:0/93502389 conn(0x7f19fc0fbc70 msgr2=0x7f19fc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- 172.21.15.96:0/93502389 shutdown_connections 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- 172.21.15.96:0/93502389 wait complete. 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 Processor -- start 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- start start 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1018d0 0x7f19fc1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 0x7f19fc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc10c360 con 0x7f19fc1006e0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc10c1e0 con 0x7f19fc102290 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a033f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc10c4e0 con 0x7f19fc1018d0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a01190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a01190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51050/0 (socket says 172.21.15.96:51050) 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.984+0000 7f1a01190640 1 -- 172.21.15.96:0/499758963 learned_addr learned my addr 172.21.15.96:0/499758963 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01991640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 0x7f19fc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a0098f640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1018d0 0x7f19fc1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 -- 172.21.15.96:0/499758963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1018d0 msgr2=0x7f19fc1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1018d0 0x7f19fc1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 -- 172.21.15.96:0/499758963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 msgr2=0x7f19fc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 0x7f19fc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 -- 172.21.15.96:0/499758963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc19f780 con 0x7f19fc1006e0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a01190640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e9c0 tx=0x7f19f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:52.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f19f27fc640 1 -- 172.21.15.96:0/499758963 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f800cd80 con 0x7f19fc1006e0 2024-03-18T15:42:52.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f19f27fc640 1 -- 172.21.15.96:0/499758963 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f80035f0 con 0x7f19fc1006e0 2024-03-18T15:42:52.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f19f27fc640 1 -- 172.21.15.96:0/499758963 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8010690 con 0x7f19fc1006e0 2024-03-18T15:42:52.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc19fa70 con 0x7f19fc1006e0 2024-03-18T15:42:52.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.985+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc19ff90 con 0x7f19fc1006e0 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.987+0000 7f19f27fc640 1 -- 172.21.15.96:0/499758963 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f19f80037a0 con 0x7f19fc1006e0 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 Processor -- start 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 -- start start 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f19f27fc640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 0x7f19e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 0x56485a879590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a881160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56485a7de500 con 0x56485a879b60 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56485a786be0 con 0x56485a879170 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56485a7da5a0 con 0x56485a87ecd0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01991640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 0x7f19e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f19f27fc640 1 -- 172.21.15.96:0/499758963 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f19f80ced60 con 0x7f19fc1006e0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a0098f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 0x56485a879590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a0098f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51058/0 (socket says 172.21.15.96:51058) 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a0098f640 1 -- 172.21.15.96:0/532075497 learned_addr learned my addr 172.21.15.96:0/532075497 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01991640 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a881160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 -- 172.21.15.96:0/532075497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 msgr2=0x56485a881160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a881160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 -- 172.21.15.96:0/532075497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 msgr2=0x56485a87e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01190640 1 -- 172.21.15.96:0/532075497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485a79a4b0 con 0x56485a879170 2024-03-18T15:42:52.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a0098f640 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.988+0000 7f1a01991640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 0x7f19e80a4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f4011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.989+0000 7f1a01190640 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 0x56485a879590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19f80cc880 tx=0x7f19f809b2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e8daf2135966a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.989+0000 7f19e77fe640 1 -- 172.21.15.96:0/532075497 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f809b6d0 con 0x56485a879170 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.989+0000 7f19e77fe640 1 -- 172.21.15.96:0/532075497 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f809f820 con 0x56485a879170 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.989+0000 7f19e77fe640 1 -- 172.21.15.96:0/532075497 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8003b10 con 0x56485a879170 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- 172.21.15.96:0/532075497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 msgr2=0x56485a879590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 0x56485a879590 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f19f80cc880 tx=0x7f19f809b2d0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- 172.21.15.96:0/532075497 shutdown_connections 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a881160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879b60 0x56485a87e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/532075497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879170 0x56485a879590 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- 172.21.15.96:0/532075497 >> 172.21.15.96:0/532075497 conn(0x56485a8746e0 msgr2=0x56485a874ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- 172.21.15.96:0/532075497 shutdown_connections 2024-03-18T15:42:52.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- 172.21.15.96:0/532075497 wait complete. 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- start start 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 Processor -- start 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a883ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a88abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56485a881c20 con 0x56485a879170 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56485a79a160 con 0x56485a879b60 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56485a881da0 con 0x56485a87ecd0 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.990+0000 7f1a05fe3740 10 client.0 _ll_register_callbacks cb 0x56485a77e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a0098f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a883ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a0098f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a883ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55316/0 (socket says 172.21.15.96:55316) 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01991640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a88abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:52.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a0098f640 1 -- 172.21.15.96:0/1116472992 learned_addr learned my addr 172.21.15.96:0/1116472992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 -- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 msgr2=0x56485a88abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a88abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 -- 172.21.15.96:0/1116472992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 msgr2=0x56485a883ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a883ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 -- 172.21.15.96:0/1116472992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485a5bc7d0 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f1a01190640 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883660 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56485a87a3c0 tx=0x7f19f809b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fc1a6e346ba49 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.991+0000 7f19e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f80b4060 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f19f8003b10 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f809f820 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19f80a3e70 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56485a4f23f0 con 0x56485a879170 2024-03-18T15:42:53.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f809f820 con 0x56485a879170 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f19f8055bb0 con 0x56485a879170 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 20 client.16224 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a88ed40 0x56485a891130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 4 client.16224 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.992+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56485a7eba40 con 0x56485a88ed40 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.993+0000 7f1a0098f640 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a88ed40 0x56485a891130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.993+0000 7f1a0098f640 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a88ed40 0x56485a891130 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf70e6f81cd9911 server_cookie=5336fb3b30752c02 in_seq=0 out_seq=0 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.993+0000 7f19e4ff9640 10 client.16224 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f19e4ff9640 1 -- 172.21.15.96:0/1116472992 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56485a7eba40 con 0x56485a88ed40 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f19e4ff9640 10 client.16224 handle_command_reply: tid=0 2024-03-18T15:42:53.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 client.16224 shutdown 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 msgr2=0x56485a883660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883660 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56485a87a3c0 tx=0x7f19f809b340 comp rx=0 tx=0).stop 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 shutdown_connections 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a87ecd0 0x56485a88abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a883ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a88ed40 0x56485a891130 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/1116472992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883660 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f19e4ff9640 0 client.16224 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:53.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 >> 172.21.15.96:0/1116472992 conn(0x56485a8746e0 msgr2=0x56485a873900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 shutdown_connections 2024-03-18T15:42:53.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 1 -- 172.21.15.96:0/1116472992 wait complete. 2024-03-18T15:42:53.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:52.994+0000 7f1a05fe3740 20 client.16224 trim_cache size 0 max 16384 2024-03-18T15:42:53.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 Processor -- start 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 -- start start 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879b60 0x56485a88ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a87ecd0 0x56485a873070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1a8d60 con 0x56485a879170 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1aa9c0 con 0x56485a87ecd0 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1a9430 con 0x56485a879b60 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51072/0 (socket says 172.21.15.96:51072) 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 -- 172.21.15.96:0/89689349 learned_addr learned my addr 172.21.15.96:0/89689349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a0098f640 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879b60 0x56485a88ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01991640 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a87ecd0 0x56485a873070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 -- 172.21.15.96:0/89689349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879b60 msgr2=0x56485a88ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879b60 0x56485a88ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 -- 172.21.15.96:0/89689349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a87ecd0 msgr2=0x56485a873070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a87ecd0 0x56485a873070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.012+0000 7f1a01190640 1 -- 172.21.15.96:0/89689349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56485a4b20f0 con 0x56485a879170 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a01190640 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f19f809b610 tx=0x7f19f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a45d476541ec7e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f19e6ffd640 1 -- 172.21.15.96:0/89689349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f80b4040 con 0x56485a879170 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f19e6ffd640 1 -- 172.21.15.96:0/89689349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f8086110 con 0x56485a879170 2024-03-18T15:42:53.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f19e6ffd640 1 -- 172.21.15.96:0/89689349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8003b10 con 0x56485a879170 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 -- 172.21.15.96:0/89689349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 msgr2=0x56485a883b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f19f809b610 tx=0x7f19f8003f10 comp rx=0 tx=0).stop 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 -- 172.21.15.96:0/89689349 shutdown_connections 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a87ecd0 0x56485a873070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879b60 0x56485a88ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/89689349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a879170 0x56485a883b00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 -- 172.21.15.96:0/89689349 >> 172.21.15.96:0/89689349 conn(0x56485a8746e0 msgr2=0x56485a874ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 -- 172.21.15.96:0/89689349 shutdown_connections 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.013+0000 7f1a05fe3740 1 -- 172.21.15.96:0/89689349 wait complete. 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 -- start start 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 Processor -- start 2024-03-18T15:42:53.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879170 0x56485a888170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a8886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1aa230 con 0x56485a87ecd0 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1a9b60 con 0x56485a879b60 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc1a6d40 con 0x56485a879170 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a05fe3740 10 client.0 _ll_register_callbacks cb 0x7f19f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a01991640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a01991640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51084/0 (socket says 172.21.15.96:51084) 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a01991640 1 -- 172.21.15.96:0/4112379322 learned_addr learned my addr 172.21.15.96:0/4112379322 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a01190640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879170 0x56485a888170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.014+0000 7f1a0098f640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a8886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 -- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879170 msgr2=0x56485a888170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879170 0x56485a888170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 -- 172.21.15.96:0/4112379322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 msgr2=0x56485a8886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a8886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 -- 172.21.15.96:0/4112379322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56485a7a59d0 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a01991640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ff80 tx=0x7f19f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756d8bb5802a03a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f401a040 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f19f4004b20 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f8086110 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19f4004d20 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 10 client.16230 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56485a88ae10 con 0x56485a87ecd0 2024-03-18T15:42:53.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f8086110 con 0x56485a87ecd0 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f19f401d030 con 0x56485a87ecd0 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 10 client.16230 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 20 client.16230 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a8f5690 0x56485a8f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 4 client.16230 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.015+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56485a7e5d90 con 0x56485a8f5690 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a01190640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a8f5690 0x56485a8f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f19e4ff9640 10 client.16230 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a01190640 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a8f5690 0x56485a8f7ab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b5ec76aa9ca5e server_cookie=f1e39815e2a1029d in_seq=0 out_seq=0 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f19e4ff9640 1 -- 172.21.15.96:0/4112379322 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56485a7e5d90 con 0x56485a8f5690 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f19e4ff9640 10 client.16230 handle_command_reply: tid=0 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 client.16230 shutdown 2024-03-18T15:42:53.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 msgr2=0x56485a889670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f19f400ff80 tx=0x7f19f4004ab0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 shutdown_connections 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56485a8f5690 0x56485a8f7ab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56485a87ecd0 0x56485a889670 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56485a879b60 0x56485a8886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f19e4ff9640 0 client.16230 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 --2- 172.21.15.96:0/4112379322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56485a879170 0x56485a888170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 >> 172.21.15.96:0/4112379322 conn(0x56485a8746e0 msgr2=0x56485a877940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 shutdown_connections 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.016+0000 7f1a05fe3740 1 -- 172.21.15.96:0/4112379322 wait complete. 2024-03-18T15:42:53.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a05fe3740 20 client.16230 trim_cache size 0 max 16384 2024-03-18T15:42:53.027 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:42:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:42:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:42:53.027 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 msgr2=0x7f19e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 0x7f19e80a4d40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f4011bc0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 msgr2=0x7f19fc197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e9c0 tx=0x7f19f800ee80 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 shutdown_connections 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f19e80a2880 0x7f19e80a4d40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f19fc102290 0x7f19fc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19fc1018d0 0x7f19fc1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 --2- 172.21.15.96:0/499758963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19fc1006e0 0x7f19fc197b60 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 >> 172.21.15.96:0/499758963 conn(0x7f19fc0fbc70 msgr2=0x7f19fc068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 shutdown_connections 2024-03-18T15:42:53.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.017+0000 7f1a033f4640 1 -- 172.21.15.96:0/499758963 wait complete. 2024-03-18T15:42:53.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:42:53.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:42:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 -- 172.21.15.96:0/4164449117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41980ff840 msgr2=0x7f41980ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 --2- 172.21.15.96:0/4164449117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41980ff840 0x7f41980ffcc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4184009900 tx=0x7f418402bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 -- 172.21.15.96:0/4164449117 shutdown_connections 2024-03-18T15:42:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 --2- 172.21.15.96:0/4164449117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4198100200 0x7f4198109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 --2- 172.21.15.96:0/4164449117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41980ff840 0x7f41980ffcc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 --2- 172.21.15.96:0/4164449117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 0x7f41981031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 -- 172.21.15.96:0/4164449117 >> 172.21.15.96:0/4164449117 conn(0x7f41980fb3e0 msgr2=0x7f41980fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 -- 172.21.15.96:0/4164449117 shutdown_connections 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.175+0000 7f419efcd640 1 -- 172.21.15.96:0/4164449117 wait complete. 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 Processor -- start 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 -- start start 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41980ff840 0x7f41981024d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 0x7f4198101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d480 con 0x7f4198100200 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d300 con 0x7f4198102dd0 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419efcd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f419810d600 con 0x7f41980ff840 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f4197fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f4197fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51092/0 (socket says 172.21.15.96:51092) 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f4197fff640 1 -- 172.21.15.96:0/3181554573 learned_addr learned my addr 172.21.15.96:0/3181554573 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419cd69640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41980ff840 0x7f41981024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.176+0000 7f419d56a640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 0x7f4198101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 -- 172.21.15.96:0/3181554573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41980ff840 msgr2=0x7f41981024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41980ff840 0x7f41981024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 -- 172.21.15.96:0/3181554573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 msgr2=0x7f4198101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 0x7f4198101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 -- 172.21.15.96:0/3181554573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41981018e0 con 0x7f4198100200 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4197fff640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f418402efd0 tx=0x7f418402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184041070 con 0x7f4198100200 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184004450 con 0x7f4198100200 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41981a1be0 con 0x7f4198100200 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184034440 con 0x7f4198100200 2024-03-18T15:42:53.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.177+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a2130 con 0x7f4198100200 2024-03-18T15:42:53.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.179+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4198108c90 con 0x7f4198100200 2024-03-18T15:42:53.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.179+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4184044070 con 0x7f4198100200 2024-03-18T15:42:53.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.180+0000 7f4195ffb640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 0x7f41880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.180+0000 7f419cd69640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 0x7f41880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.180+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f41840f0890 con 0x7f4198100200 2024-03-18T15:42:53.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.180+0000 7f419cd69640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 0x7f41880a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4198068660 tx=0x7f418c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.185+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f41840bd2b0 con 0x7f4198100200 2024-03-18T15:42:53.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.426+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41980623e0 con 0x7f4198100200 2024-03-18T15:42:53.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.426+0000 7f4195ffb640 1 -- 172.21.15.96:0/3181554573 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11945 (secure 0 0 0) 0x7f41840b7e20 con 0x7f4198100200 2024-03-18T15:42:53.427 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:42:53.427 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:41:28.657174+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:42:53.428 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 29 2024-03-18T15:42:53.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 msgr2=0x7f41880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 0x7f41880a4cc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4198068660 tx=0x7f418c005f90 comp rx=0 tx=0).stop 2024-03-18T15:42:53.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 msgr2=0x7f4198100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f418402efd0 tx=0x7f418402c680 comp rx=0 tx=0).stop 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 shutdown_connections 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41880a2800 0x7f41880a4cc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4198102dd0 0x7f4198101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198100200 0x7f4198100b10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 --2- 172.21.15.96:0/3181554573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41980ff840 0x7f41981024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.428+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 >> 172.21.15.96:0/3181554573 conn(0x7f41980fb3e0 msgr2=0x7f4198105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.429+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 shutdown_connections 2024-03-18T15:42:53.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.429+0000 7f419efcd640 1 -- 172.21.15.96:0/3181554573 wait complete. 2024-03-18T15:42:53.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:42:53.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 -- 172.21.15.96:0/2843703908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 msgr2=0x7f9a94100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 --2- 172.21.15.96:0/2843703908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94100510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009900 tx=0x7f9a8402bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 -- 172.21.15.96:0/2843703908 shutdown_connections 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 --2- 172.21.15.96:0/2843703908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a94101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 --2- 172.21.15.96:0/2843703908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 0x7f9a94100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 --2- 172.21.15.96:0/2843703908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94100510 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 -- 172.21.15.96:0/2843703908 >> 172.21.15.96:0/2843703908 conn(0x7f9a940fbc90 msgr2=0x7f9a940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 -- 172.21.15.96:0/2843703908 shutdown_connections 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 -- 172.21.15.96:0/2843703908 wait complete. 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.553+0000 7f9a99463640 1 Processor -- start 2024-03-18T15:42:53.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 -- start start 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 0x7f9a9419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a941a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410f6d0 con 0x7f9a941000f0 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410f550 con 0x7f9a941013f0 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a99463640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9410f850 con 0x7f9a94100a50 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a92ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a941a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a92ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56140/0 (socket says 172.21.15.96:56140) 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a92ffd640 1 -- 172.21.15.96:0/210417319 learned_addr learned my addr 172.21.15.96:0/210417319 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a927fc640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 0x7f9a9419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 -- 172.21.15.96:0/210417319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 msgr2=0x7f9a9419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 0x7f9a9419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 -- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 msgr2=0x7f9a94199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.554+0000 7f9a937fe640 1 -- 172.21.15.96:0/210417319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a941a19f0 con 0x7f9a941013f0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a937fe640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a941a1380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009fd0 tx=0x7f9a8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a7bfff640 1 -- 172.21.15.96:0/210417319 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00ce20 con 0x7f9a941013f0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941a1ce0 con 0x7f9a941013f0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a7bfff640 1 -- 172.21.15.96:0/210417319 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a8c005020 con 0x7f9a941013f0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a7bfff640 1 -- 172.21.15.96:0/210417319 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c018760 con 0x7f9a941013f0 2024-03-18T15:42:53.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.555+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941a21d0 con 0x7f9a941013f0 2024-03-18T15:42:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a7bfff640 1 -- 172.21.15.96:0/210417319 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f9a8c018900 con 0x7f9a941013f0 2024-03-18T15:42:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 Processor -- start 2024-03-18T15:42:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 -- start start 2024-03-18T15:42:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac111d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac114840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac071490 con 0x55e6ac1123b0 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac019c90 con 0x55e6ac10c850 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac06d530 con 0x55e6ac10d240 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a92ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac111d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:59866/0 (socket says 172.21.15.96:59866) 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.558+0000 7f9a927fc640 1 -- 172.21.15.96:0/1784646152 learned_addr learned my addr 172.21.15.96:0/1784646152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac114840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a7bfff640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 0x7f9a7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 -- 172.21.15.96:0/1784646152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 msgr2=0x55e6ac111d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac111d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 -- 172.21.15.96:0/1784646152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 msgr2=0x55e6ac114840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac114840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 -- 172.21.15.96:0/1784646152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ac0d4500 con 0x55e6ac10c850 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a7bfff640 1 -- 172.21.15.96:0/210417319 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9a8c0cf150 con 0x7f9a941013f0 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a937fe640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 0x7f9a7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a927fc640 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9a88011810 tx=0x7f9a880097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7c7203085183ef server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a797fa640 1 -- 172.21.15.96:0/1784646152 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88004030 con 0x55e6ac10c850 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a797fa640 1 -- 172.21.15.96:0/1784646152 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88011e50 con 0x55e6ac10c850 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.559+0000 7f9a797fa640 1 -- 172.21.15.96:0/1784646152 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88003850 con 0x55e6ac10c850 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 -- 172.21.15.96:0/1784646152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 msgr2=0x55e6ac10cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9a88011810 tx=0x7f9a880097d0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a937fe640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 0x7f9a7c0a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005b40 tx=0x7f9a8c00c890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 -- 172.21.15.96:0/1784646152 shutdown_connections 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac114840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac111d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/1784646152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac10cc70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 -- 172.21.15.96:0/1784646152 >> 172.21.15.96:0/1784646152 conn(0x55e6ac107dc0 msgr2=0x55e6ac1081c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 -- 172.21.15.96:0/1784646152 shutdown_connections 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.560+0000 7f9a9c04a740 1 -- 172.21.15.96:0/1784646152 wait complete. 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 -- start start 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 Processor -- start 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac116d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac1172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac115360 con 0x55e6ac1123b0 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac02d230 con 0x55e6ac10c850 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e6ac1154e0 con 0x55e6ac10d240 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a9c04a740 10 client.0 _ll_register_callbacks cb 0x55e6ac069010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac1172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56158/0 (socket says 172.21.15.96:56158) 2024-03-18T15:42:53.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a927fc640 1 -- 172.21.15.96:0/2789277674 learned_addr learned my addr 172.21.15.96:0/2789277674 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a927fc640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac116d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 -- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 msgr2=0x55e6ac116d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac116d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 -- 172.21.15.96:0/2789277674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 msgr2=0x55e6ac11e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a92ffd640 1 -- 172.21.15.96:0/2789277674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ac03c100 con 0x55e6ac10d240 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.561+0000 7f9a927fc640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac1172c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a84005ab0 tx=0x7f9a84005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36de7be4b5b6bac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004390 con 0x55e6ac10d240 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a88011e50 con 0x55e6ac10d240 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9a840046c0 con 0x55e6ac10d240 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a840048c0 con 0x55e6ac10d240 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a9c04a740 10 client.26092 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.562+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6abd853f0 con 0x55e6ac10d240 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88011e50 con 0x55e6ac10d240 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f9a840035f0 con 0x55e6ac10d240 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 10 client.26092 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 10 client.26092 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 20 client.26092 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac122bd0 0x55e6ac124ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 4 client.26092 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6ac07e9d0 con 0x55e6ac122bd0 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a927fc640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac122bd0 0x55e6ac124ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a76ff5640 10 client.26092 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.563+0000 7f9a927fc640 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac122bd0 0x55e6ac124ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1ee664f954453df8 server_cookie=8b46478538ebd44 in_seq=0 out_seq=0 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a76ff5640 1 -- 172.21.15.96:0/2789277674 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e6ac07e9d0 con 0x55e6ac122bd0 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a76ff5640 10 client.26092 handle_command_reply: tid=0 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 client.26092 shutdown 2024-03-18T15:42:53.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 msgr2=0x55e6ac1172c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac1172c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a84005ab0 tx=0x7f9a84005d50 comp rx=0 tx=0).stop 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 shutdown_connections 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac122bd0 0x55e6ac124ff0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac1123b0 0x55e6ac11e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac1172c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a76ff5640 0 client.26092 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/2789277674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac10c850 0x55e6ac116d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 >> 172.21.15.96:0/2789277674 conn(0x55e6ac107dc0 msgr2=0x55e6ac106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 shutdown_connections 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 1 -- 172.21.15.96:0/2789277674 wait complete. 2024-03-18T15:42:53.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.564+0000 7f9a9c04a740 20 client.26092 trim_cache size 0 max 16384 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 Processor -- start 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 -- start start 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac106c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941aafc0 con 0x55e6ac10c850 2024-03-18T15:42:53.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941acc20 con 0x55e6ac1123b0 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.579+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941ab690 con 0x55e6ac10d240 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56170/0 (socket says 172.21.15.96:56170) 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 -- 172.21.15.96:0/472667641 learned_addr learned my addr 172.21.15.96:0/472667641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 -- 172.21.15.96:0/472667641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 msgr2=0x55e6ac11e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 -- 172.21.15.96:0/472667641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 msgr2=0x55e6ac106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 -- 172.21.15.96:0/472667641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ac125360 con 0x55e6ac10c850 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a927fc640 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9a88004650 tx=0x7f9a88005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29bd4fd9f0cb24c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a78ff9640 1 -- 172.21.15.96:0/472667641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8800fd80 con 0x55e6ac10c850 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a78ff9640 1 -- 172.21.15.96:0/472667641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88011e50 con 0x55e6ac10c850 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.580+0000 7f9a78ff9640 1 -- 172.21.15.96:0/472667641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88005420 con 0x55e6ac10c850 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- 172.21.15.96:0/472667641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 msgr2=0x55e6ac106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9a88004650 tx=0x7f9a88005f90 comp rx=0 tx=0).stop 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- 172.21.15.96:0/472667641 shutdown_connections 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/472667641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac106750 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- 172.21.15.96:0/472667641 >> 172.21.15.96:0/472667641 conn(0x55e6ac107dc0 msgr2=0x55e6ac116d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- 172.21.15.96:0/472667641 shutdown_connections 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- 172.21.15.96:0/472667641 wait complete. 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 -- start start 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.581+0000 7f9a9c04a740 1 Processor -- start 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac11c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941ac490 con 0x55e6ac10c850 2024-03-18T15:42:53.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941abdc0 con 0x55e6ac1123b0 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a941a8fa0 con 0x55e6ac10d240 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 10 client.0 _ll_register_callbacks cb 0x7f9a84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56176/0 (socket says 172.21.15.96:56176) 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 -- 172.21.15.96:0/3560013900 learned_addr learned my addr 172.21.15.96:0/3560013900 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac11c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 -- 172.21.15.96:0/3560013900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 msgr2=0x55e6ac11b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 -- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 msgr2=0x55e6ac11c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac11c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 -- 172.21.15.96:0/3560013900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6abe4f7d0 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a927fc640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9a88011040 tx=0x7f9a880115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2507d509b671e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880111a0 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f9a880303d0 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a88011e50 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a880305d0 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.582+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6ac11e100 con 0x55e6ac10c850 2024-03-18T15:42:53.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a88011e50 con 0x55e6ac10c850 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f9a88047070 con 0x55e6ac10c850 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 10 client.16254 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 20 client.16254 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac188fb0 0x55e6ac18b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 4 client.16254 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6ac116b70 con 0x55e6ac188fb0 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac188fb0 0x55e6ac18b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a76ff5640 10 client.16254 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a92ffd640 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac188fb0 0x55e6ac18b3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3991611108b28043 server_cookie=5d2169250477b9d2 in_seq=0 out_seq=0 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.583+0000 7f9a76ff5640 1 -- 172.21.15.96:0/3560013900 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6ac116b70 con 0x55e6ac188fb0 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a76ff5640 10 client.16254 handle_command_reply: tid=0 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 client.16254 shutdown 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 msgr2=0x55e6ac11b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9a88011040 tx=0x7f9a880115b0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 shutdown_connections 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e6ac188fb0 0x55e6ac18b3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e6ac1123b0 0x55e6ac11c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a76ff5640 0 client.16254 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e6ac10d240 0x55e6ac11b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 --2- 172.21.15.96:0/3560013900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6ac10c850 0x55e6ac11b490 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 >> 172.21.15.96:0/3560013900 conn(0x55e6ac107dc0 msgr2=0x55e6ac10b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 shutdown_connections 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 1 -- 172.21.15.96:0/3560013900 wait complete. 2024-03-18T15:42:53.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a9c04a740 20 client.16254 trim_cache size 0 max 16384 2024-03-18T15:42:53.595 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 msgr2=0x7f9a7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 0x7f9a7c0a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005b40 tx=0x7f9a8c00c890 comp rx=0 tx=0).stop 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 msgr2=0x7f9a941a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a941a1380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009fd0 tx=0x7f9a8c00ef10 comp rx=0 tx=0).stop 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 shutdown_connections 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9a7c0a2800 0x7f9a7c0a4cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a941013f0 0x7f9a941a1380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a94100a50 0x7f9a9419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 --2- 172.21.15.96:0/210417319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a941000f0 0x7f9a94199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.584+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 >> 172.21.15.96:0/210417319 conn(0x7f9a940fbc90 msgr2=0x7f9a940fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.585+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 shutdown_connections 2024-03-18T15:42:53.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.585+0000 7f9a99463640 1 -- 172.21.15.96:0/210417319 wait complete. 2024-03-18T15:42:53.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.706+0000 7f55f10de640 1 -- 172.21.15.96:0/4274241520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 msgr2=0x7f55ec100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.706+0000 7f55f10de640 1 --2- 172.21.15.96:0/4274241520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec100b40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 -- 172.21.15.96:0/4274241520 shutdown_connections 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 --2- 172.21.15.96:0/4274241520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 --2- 172.21.15.96:0/4274241520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 0x7f55ec101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 --2- 172.21.15.96:0/4274241520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec100b40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 -- 172.21.15.96:0/4274241520 >> 172.21.15.96:0/4274241520 conn(0x7f55ec0fbc90 msgr2=0x7f55ec0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 -- 172.21.15.96:0/4274241520 shutdown_connections 2024-03-18T15:42:53.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 -- 172.21.15.96:0/4274241520 wait complete. 2024-03-18T15:42:53.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.707+0000 7f55f10de640 1 Processor -- start 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 -- start start 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 0x7f55ec102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10c370 con 0x7f55ec100720 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10c1f0 con 0x7f55ec1022d0 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55f10de640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec10c4f0 con 0x7f55ec101910 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56194/0 (socket says 172.21.15.96:56194) 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 -- 172.21.15.96:0/1796514539 learned_addr learned my addr 172.21.15.96:0/1796514539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 0x7f55ec102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 -- 172.21.15.96:0/1796514539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 msgr2=0x7f55ec102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 0x7f55ec102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 -- 172.21.15.96:0/1796514539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 msgr2=0x7f55ec103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 -- 172.21.15.96:0/1796514539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ec103ac0 con 0x7f55ec100720 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.708+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000c00 tx=0x7f55dc004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.709+0000 7f55d3fff640 1 -- 172.21.15.96:0/1796514539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc041070 con 0x7f55ec100720 2024-03-18T15:42:53.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.709+0000 7f55d3fff640 1 -- 172.21.15.96:0/1796514539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55dc004450 con 0x7f55ec100720 2024-03-18T15:42:53.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.709+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ec1a1c90 con 0x7f55ec100720 2024-03-18T15:42:53.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.709+0000 7f55d3fff640 1 -- 172.21.15.96:0/1796514539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc003d90 con 0x7f55ec100720 2024-03-18T15:42:53.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.709+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ec1a2190 con 0x7f55ec100720 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55d3fff640 1 -- 172.21.15.96:0/1796514539 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f55dc044070 con 0x7f55ec100720 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 Processor -- start 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 -- start start 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf0850 0x559227cf0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 0x559227cf8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559227c55490 con 0x559227cf1240 2024-03-18T15:42:53.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559227bfdc90 con 0x559227cf63b0 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.711+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559227c51530 con 0x559227cf0850 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55eb59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf0850 0x559227cf0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56204/0 (socket says 172.21.15.96:56204) 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ead9d640 1 -- 172.21.15.96:0/1737486851 learned_addr learned my addr 172.21.15.96:0/1737486851 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 0x559227cf8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55d3fff640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 0x7f55d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 0x7f55d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 -- 172.21.15.96:0/1737486851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf0850 msgr2=0x559227cf0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf0850 0x559227cf0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 -- 172.21.15.96:0/1737486851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 msgr2=0x559227cf5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 -- 172.21.15.96:0/1737486851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559227cb8500 con 0x559227cf63b0 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55d3fff640 1 -- 172.21.15.96:0/1796514539 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f55dc0f0510 con 0x7f55ec100720 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 0x559227cf8840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009bf0 tx=0x7f55e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb844c8ce792860 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 0x7f55d40a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x559227cf1aa0 tx=0x7f55e4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55d17fa640 1 -- 172.21.15.96:0/1737486851 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004050 con 0x559227cf63b0 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55d17fa640 1 -- 172.21.15.96:0/1737486851 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0005500 con 0x559227cf63b0 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.712+0000 7f55d17fa640 1 -- 172.21.15.96:0/1737486851 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004e00 con 0x559227cf63b0 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1737486851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 msgr2=0x559227cf8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 0x559227cf8840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009bf0 tx=0x7f55e0011810 comp rx=0 tx=0).stop 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1737486851 shutdown_connections 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf63b0 0x559227cf8840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1737486851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf0850 0x559227cf0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1737486851 >> 172.21.15.96:0/1737486851 conn(0x559227cebdc0 msgr2=0x559227cec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1737486851 shutdown_connections 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1737486851 wait complete. 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.713+0000 7f55f3d9b740 1 -- start start 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 Processor -- start 2024-03-18T15:42:53.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf1240 0x559227cfb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559227cf9360 con 0x559227cf0850 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559227c11230 con 0x559227cf1240 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559227cf94e0 con 0x559227cf63b0 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 10 client.0 _ll_register_callbacks cb 0x559227c4d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56216/0 (socket says 172.21.15.96:56216) 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 -- 172.21.15.96:0/1176456289 learned_addr learned my addr 172.21.15.96:0/1176456289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf1240 0x559227cfb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 -- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 msgr2=0x559227d02260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 -- 172.21.15.96:0/1176456289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf1240 msgr2=0x559227cfb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf1240 0x559227cfb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 -- 172.21.15.96:0/1176456289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559227c20100 con 0x559227cf0850 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ee030 tx=0x7f55dc0f3ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b904d474f4351b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc04b040 con 0x559227cf0850 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f55dc0b94f0 con 0x559227cf0850 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.714+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e0005500 con 0x559227cf0850 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.715+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55dc0bc7e0 con 0x559227cf0850 2024-03-18T15:42:53.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.715+0000 7f55f3d9b740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.715+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592279693f0 con 0x559227cf0850 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.715+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0005500 con 0x559227cf0850 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.715+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f55dc0c5190 con 0x559227cf0850 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 10 client.16266 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 20 client.16266 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d06af0 0x559227d08f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 4 client.16266 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559227c629d0 con 0x559227d06af0 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.716+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d06af0 0x559227d08f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.717+0000 7f55ceff5640 10 client.16266 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.717+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d06af0 0x559227d08f60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22e46572f5efd355 server_cookie=5a51091293127a16 in_seq=0 out_seq=0 2024-03-18T15:42:53.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55ceff5640 1 -- 172.21.15.96:0/1176456289 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559227c629d0 con 0x559227d06af0 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55ceff5640 10 client.16266 handle_command_reply: tid=0 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 client.16266 shutdown 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 msgr2=0x559227cfad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ee030 tx=0x7f55dc0f3ce0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 shutdown_connections 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d06af0 0x559227d08f60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55ceff5640 0 client.16266 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf1240 0x559227cfb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1176456289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf0850 0x559227cfad20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 >> 172.21.15.96:0/1176456289 conn(0x559227cebdc0 msgr2=0x559227ceafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 shutdown_connections 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1176456289 wait complete. 2024-03-18T15:42:53.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.718+0000 7f55f3d9b740 20 client.16266 trim_cache size 0 max 16384 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.734+0000 7f55f3d9b740 1 Processor -- start 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.734+0000 7f55f3d9b740 1 -- start start 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227cf2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1ab060 con 0x559227cf1240 2024-03-18T15:42:53.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1accc0 con 0x559227cf0850 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1ab730 con 0x559227cf63b0 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56220/0 (socket says 172.21.15.96:56220) 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 -- 172.21.15.96:0/1593786998 learned_addr learned my addr 172.21.15.96:0/1593786998 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ead9d640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227cf2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 -- 172.21.15.96:0/1593786998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 msgr2=0x559227d02220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 -- 172.21.15.96:0/1593786998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 msgr2=0x559227cf2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227cf2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 -- 172.21.15.96:0/1593786998 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559227a337d0 con 0x559227cf1240 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55eb59e640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55ea59c640 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x559227cecd30 tx=0x7f55e00097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192fdf28c708d857 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55d0ff9640 1 -- 172.21.15.96:0/1593786998 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0009810 con 0x559227cf1240 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55d0ff9640 1 -- 172.21.15.96:0/1593786998 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0004520 con 0x559227cf1240 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.735+0000 7f55d0ff9640 1 -- 172.21.15.96:0/1593786998 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e002e480 con 0x559227cf1240 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1593786998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 msgr2=0x559227cf2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x559227cecd30 tx=0x7f55e00097a0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1593786998 shutdown_connections 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227d02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cf2c40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/1593786998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227cf2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1593786998 >> 172.21.15.96:0/1593786998 conn(0x559227cebdc0 msgr2=0x559227cec1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1593786998 shutdown_connections 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- 172.21.15.96:0/1593786998 wait complete. 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.736+0000 7f55f3d9b740 1 -- start start 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 Processor -- start 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227d01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227cfefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1ac530 con 0x559227cf1240 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1abe60 con 0x559227cf0850 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55ec1a9040 con 0x559227cf63b0 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55f3d9b740 10 client.0 _ll_register_callbacks cb 0x7f55dc005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227d01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56228/0 (socket says 172.21.15.96:56228) 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ea59c640 1 -- 172.21.15.96:0/3436860414 learned_addr learned my addr 172.21.15.96:0/3436860414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55eb59e640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227cfefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 -- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 msgr2=0x559227cfefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227cfefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 -- 172.21.15.96:0/3436860414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 msgr2=0x559227cfea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 -- 172.21.15.96:0/3436860414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559227cff610 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ea59c640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ead9d640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227d01f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0bce10 tx=0x7f55dc0ee060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647fef2c846097b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.737+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc04b070 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55e0004520 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f55dc0f0150 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f55dc0c5670 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 10 client.26063 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559227d6bf90 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0004520 con 0x559227cf0850 2024-03-18T15:42:53.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f55dc0bc2d0 con 0x559227cf0850 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 10 client.26063 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 10 client.26063 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 20 client.26063 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d6c530 0x559227d6e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 4 client.26063 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559227c5cd20 con 0x559227d6c530 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.738+0000 7f55ea59c640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d6c530 0x559227d6e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.739+0000 7f55ceff5640 10 client.26063 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.739+0000 7f55ea59c640 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d6c530 0x559227d6e970 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e909c3535bea419 server_cookie=72b058fde485714f in_seq=0 out_seq=0 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55ceff5640 1 -- 172.21.15.96:0/3436860414 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559227c5cd20 con 0x559227d6c530 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55ceff5640 10 client.26063 handle_command_reply: tid=0 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 client.26063 shutdown 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 msgr2=0x559227d01f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227d01f70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0bce10 tx=0x7f55dc0ee060 comp rx=0 tx=0).stop 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 shutdown_connections 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559227d6c530 0x559227d6e970 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559227cf63b0 0x559227cfefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55ceff5640 0 client.26063 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559227cf1240 0x559227cfea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 --2- 172.21.15.96:0/3436860414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559227cf0850 0x559227d01f70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 >> 172.21.15.96:0/3436860414 conn(0x559227cebdc0 msgr2=0x559227cf9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 shutdown_connections 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 1 -- 172.21.15.96:0/3436860414 wait complete. 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f3d9b740 20 client.26063 trim_cache size 0 max 16384 2024-03-18T15:42:53.752 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:42:53.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 msgr2=0x7f55d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 0x7f55d40a4cc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x559227cf1aa0 tx=0x7f55e4004ab0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 msgr2=0x7f55ec1046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f55dc000c00 tx=0x7f55dc004960 comp rx=0 tx=0).stop 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 shutdown_connections 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f55d40a2800 0x7f55d40a4cc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.740+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55ec1022d0 0x7f55ec103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.741+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55ec101910 0x7f55ec102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.741+0000 7f55f10de640 1 --2- 172.21.15.96:0/1796514539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55ec100720 0x7f55ec1046b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.741+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 >> 172.21.15.96:0/1796514539 conn(0x7f55ec0fbc90 msgr2=0x7f55ec069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.741+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 shutdown_connections 2024-03-18T15:42:53.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.741+0000 7f55f10de640 1 -- 172.21.15.96:0/1796514539 wait complete. 2024-03-18T15:42:53.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:53 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3181554573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:42:53.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:42:53.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:53 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3181554573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.863+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1466709493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 msgr2=0x7f1cc4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.863+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1466709493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4101d50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009900 tx=0x7f1cb002bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1466709493 shutdown_connections 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1466709493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 0x7f1cc4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1466709493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4101d50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1466709493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 0x7f1cc4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1466709493 >> 172.21.15.96:0/1466709493 conn(0x7f1cc40fbc70 msgr2=0x7f1cc40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1466709493 shutdown_connections 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1466709493 wait complete. 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 Processor -- start 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- start start 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 0x7f1cc4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 0x7f1cc419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410c360 con 0x7f1cc41018d0 2024-03-18T15:42:53.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410c1e0 con 0x7f1cc4102290 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1ccafd4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc410c4e0 con 0x7f1cc41006e0 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56244/0 (socket says 172.21.15.96:56244) 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1cc3fff640 1 -- 172.21.15.96:0/1460943414 learned_addr learned my addr 172.21.15.96:0/1460943414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 0x7f1cc4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.864+0000 7f1cc9571640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 0x7f1cc419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 -- 172.21.15.96:0/1460943414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 msgr2=0x7f1cc4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 0x7f1cc4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 -- 172.21.15.96:0/1460943414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 msgr2=0x7f1cc419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 0x7f1cc419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 -- 172.21.15.96:0/1460943414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc419e670 con 0x7f1cc41018d0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009230 tx=0x7f1cb002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc1ffb640 1 -- 172.21.15.96:0/1460943414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0041070 con 0x7f1cc41018d0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc419e900 con 0x7f1cc41018d0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc1ffb640 1 -- 172.21.15.96:0/1460943414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0004450 con 0x7f1cc41018d0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1cc1ffb640 1 -- 172.21.15.96:0/1460943414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0003e20 con 0x7f1cc41018d0 2024-03-18T15:42:53.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.865+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc419edf0 con 0x7f1cc41018d0 2024-03-18T15:42:53.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.867+0000 7f1cc1ffb640 1 -- 172.21.15.96:0/1460943414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1cb0044070 con 0x7f1cc41018d0 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 Processor -- start 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 -- start start 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc1ffb640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 0x7f1cb40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b52f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561509ab0490 con 0x561509b535c0 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561509a58c90 con 0x561509b4da60 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561509aac530 con 0x561509b4e450 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 0x7f1cb40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc1ffb640 1 -- 172.21.15.96:0/1460943414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1cb00f0620 con 0x7f1cc41018d0 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56260/0 (socket says 172.21.15.96:56260) 2024-03-18T15:42:53.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:59978/0 (socket says 172.21.15.96:59978) 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 -- 172.21.15.96:0/3453468614 learned_addr learned my addr 172.21.15.96:0/3453468614 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b52f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 -- 172.21.15.96:0/3453468614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 msgr2=0x561509b52f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b52f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 -- 172.21.15.96:0/3453468614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 msgr2=0x561509b4de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc9571640 1 -- 172.21.15.96:0/3453468614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561509b13500 con 0x561509b535c0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.868+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b52f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.869+0000 7f1cc9571640 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc011e10 tx=0x7f1cbc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebe963c69f9cac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.869+0000 7f1cc9571640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 0x7f1cb40a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00fc90 tx=0x7f1cbc018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.869+0000 7f1caeffd640 1 -- 172.21.15.96:0/3453468614 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc023070 con 0x561509b535c0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.869+0000 7f1caeffd640 1 -- 172.21.15.96:0/3453468614 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00ec50 con 0x561509b535c0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.869+0000 7f1caeffd640 1 -- 172.21.15.96:0/3453468614 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc005780 con 0x561509b535c0 2024-03-18T15:42:53.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/3453468614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 msgr2=0x561509b55a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc011e10 tx=0x7f1cbc00f790 comp rx=0 tx=0).stop 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/3453468614 shutdown_connections 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b55a50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b52f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/3453468614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/3453468614 >> 172.21.15.96:0/3453468614 conn(0x561509b48fd0 msgr2=0x561509b493d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/3453468614 shutdown_connections 2024-03-18T15:42:53.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/3453468614 wait complete. 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- start start 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 Processor -- start 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4e450 0x561509b584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561509b56510 con 0x561509b4da60 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561509a6c230 con 0x561509b4e450 2024-03-18T15:42:53.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561509b56690 con 0x561509b535c0 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1ccdbc0740 10 client.0 _ll_register_callbacks cb 0x561509aa8010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.870+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56266/0 (socket says 172.21.15.96:56266) 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 -- 172.21.15.96:0/1513204968 learned_addr learned my addr 172.21.15.96:0/1513204968 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc9571640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4e450 0x561509b584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 -- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 msgr2=0x561509b5f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 -- 172.21.15.96:0/1513204968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4e450 msgr2=0x561509b584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4e450 0x561509b584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 -- 172.21.15.96:0/1513204968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561509a7b100 con 0x561509b4da60 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8012700 tx=0x7f1cb8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e715da84fb26f89e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8004030 con 0x561509b4da60 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1cb80041d0 con 0x561509b4da60 2024-03-18T15:42:53.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc00ec50 con 0x561509b4da60 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cb80043d0 con 0x561509b4da60 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ccdbc0740 10 client.16284 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615097c43f0 con 0x561509b4da60 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.871+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00ec50 con 0x561509b4da60 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f1cb8025070 con 0x561509b4da60 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 10 client.16284 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 10 client.16284 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:42:53.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 20 client.16284 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509b63e80 0x561509b66320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 4 client.16284 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561509abd9d0 con 0x561509b63e80 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509b63e80 0x561509b66320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1ca3fff640 10 client.16284 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.872+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509b63e80 0x561509b66320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc48dc2ad5bd0777 server_cookie=619df6e28d00f159 in_seq=0 out_seq=0 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ca3fff640 1 -- 172.21.15.96:0/1513204968 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561509abd9d0 con 0x561509b63e80 2024-03-18T15:42:53.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ca3fff640 10 client.16284 handle_command_reply: tid=0 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 client.16284 shutdown 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 msgr2=0x561509b57f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8012700 tx=0x7f1cb8012bc0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 shutdown_connections 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509b63e80 0x561509b66320 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ca3fff640 0 client.16284 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4e450 0x561509b584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/1513204968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4da60 0x561509b57f70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 >> 172.21.15.96:0/1513204968 conn(0x561509b48fd0 msgr2=0x561509b45fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 shutdown_connections 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.874+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/1513204968 wait complete. 2024-03-18T15:42:53.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.875+0000 7f1ccdbc0740 20 client.16284 trim_cache size 0 max 16384 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 Processor -- start 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 -- start start 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b5f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b57240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a6b40 con 0x561509b535c0 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a87a0 con 0x561509b4da60 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a7210 con 0x561509b4e450 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b57240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b5f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60000/0 (socket says 172.21.15.96:60000) 2024-03-18T15:42:53.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b5f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:59760/0 (socket says 172.21.15.96:59760) 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc8d70640 1 -- 172.21.15.96:0/222324021 learned_addr learned my addr 172.21.15.96:0/222324021 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 -- 172.21.15.96:0/222324021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 msgr2=0x561509b5f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b5f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 -- 172.21.15.96:0/222324021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 msgr2=0x561509b4c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc9571640 1 -- 172.21.15.96:0/222324021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56150988e7d0 con 0x561509b535c0 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.889+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.890+0000 7f1cc9571640 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b57240 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009540 tx=0x7f1cbc00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6265995a6f01d55 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.890+0000 7f1cae7fc640 1 -- 172.21.15.96:0/222324021 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc023040 con 0x561509b535c0 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.890+0000 7f1cae7fc640 1 -- 172.21.15.96:0/222324021 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00ec50 con 0x561509b535c0 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.890+0000 7f1cae7fc640 1 -- 172.21.15.96:0/222324021 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc003cf0 con 0x561509b535c0 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/222324021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 msgr2=0x561509b57240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b57240 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009540 tx=0x7f1cbc00ff60 comp rx=0 tx=0).stop 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/222324021 shutdown_connections 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b535c0 0x561509b57240 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b4e450 0x561509b5f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/222324021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b4c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/222324021 >> 172.21.15.96:0/222324021 conn(0x561509b48fd0 msgr2=0x561509b4c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/222324021 shutdown_connections 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/222324021 wait complete. 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- start start 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 Processor -- start 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b5c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a8010 con 0x561509b4e450 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a7940 con 0x561509b4da60 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.891+0000 7f1ccdbc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41a4b20 con 0x561509b535c0 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ccdbc0740 10 client.0 _ll_register_callbacks cb 0x7f1cb8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc8d70640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b5c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56276/0 (socket says 172.21.15.96:56276) 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 -- 172.21.15.96:0/17587225 learned_addr learned my addr 172.21.15.96:0/17587225 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc9571640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 -- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 msgr2=0x561509b5db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 -- 172.21.15.96:0/17587225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 msgr2=0x561509b5c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:53.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b5c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 -- 172.21.15.96:0/17587225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561509b5e170 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b5c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1cc3fff640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b94a0 tx=0x7f1cb002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e84ab1fb5ec3ecd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb004b040 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1cb00a41b0 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc00ec50 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1cb00c5730 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ccdbc0740 10 client.16290 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561509b5f540 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.892+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00ec50 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f1cb00c52a0 con 0x561509b4e450 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 10 client.16290 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:42:53.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 20 client.16290 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509bca730 0x561509bccb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 4 client.16290 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561509ab7d20 con 0x561509bca730 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509bca730 0x561509bccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1ca3fff640 10 client.16290 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.893+0000 7f1cc8d70640 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509bca730 0x561509bccb50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6a7e11194b5bbb2 server_cookie=c83db0eb73610476 in_seq=0 out_seq=0 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ca3fff640 1 -- 172.21.15.96:0/17587225 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561509ab7d20 con 0x561509bca730 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ca3fff640 10 client.16290 handle_command_reply: tid=0 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 client.16290 shutdown 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 msgr2=0x561509b5cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1cb00b94a0 tx=0x7f1cb002bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 shutdown_connections 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x561509bca730 0x561509bccb50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561509b535c0 0x561509b5db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561509b4e450 0x561509b5cb70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ca3fff640 0 client.16290 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 --2- 172.21.15.96:0/17587225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561509b4da60 0x561509b5c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 >> 172.21.15.96:0/17587225 conn(0x561509b48fd0 msgr2=0x561509b4cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 shutdown_connections 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 1 -- 172.21.15.96:0/17587225 wait complete. 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.896+0000 7f1ccdbc0740 20 client.16290 trim_cache size 0 max 16384 2024-03-18T15:42:53.913 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 msgr2=0x7f1cb40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 0x7f1cb40a4cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00fc90 tx=0x7f1cbc018000 comp rx=0 tx=0).stop 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 msgr2=0x7f1cc4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009230 tx=0x7f1cb002c650 comp rx=0 tx=0).stop 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 shutdown_connections 2024-03-18T15:42:53.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1cb40a2800 0x7f1cb40a4cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1cc4102290 0x7f1cc419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41018d0 0x7f1cc4196f90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 --2- 172.21.15.96:0/1460943414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cc41006e0 0x7f1cc4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 >> 172.21.15.96:0/1460943414 conn(0x7f1cc40fbc70 msgr2=0x7f1cc4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 shutdown_connections 2024-03-18T15:42:53.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:53.897+0000 7f1ccafd4640 1 -- 172.21.15.96:0/1460943414 wait complete. 2024-03-18T15:42:53.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:42:53.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:53 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3181554573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:42:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 -- 172.21.15.96:0/327965845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48104b90 msgr2=0x7f8a48106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/327965845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48104b90 0x7f8a48106f80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009900 tx=0x7f8a3002bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 -- 172.21.15.96:0/327965845 shutdown_connections 2024-03-18T15:42:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/327965845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a481074c0 0x7f8a48109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/327965845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48104b90 0x7f8a48106f80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/327965845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a48102260 0x7f8a48104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 -- 172.21.15.96:0/327965845 >> 172.21.15.96:0/327965845 conn(0x7f8a480fbc70 msgr2=0x7f8a480fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 -- 172.21.15.96:0/327965845 shutdown_connections 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.023+0000 7f8a4d04b640 1 -- 172.21.15.96:0/327965845 wait complete. 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 Processor -- start 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 -- start start 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a481074c0 0x7f8a48100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d470 con 0x7f8a48104b90 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d2f0 con 0x7f8a48102260 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4d04b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d5f0 con 0x7f8a481074c0 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a46d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56298/0 (socket says 172.21.15.96:56298) 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a46d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60034/0 (socket says 172.21.15.96:60034) 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a3ffff640 1 -- 172.21.15.96:0/434145433 learned_addr learned my addr 172.21.15.96:0/434145433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.024+0000 7f8a4759e640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a481074c0 0x7f8a48100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 -- 172.21.15.96:0/434145433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a481074c0 msgr2=0x7f8a48100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a481074c0 0x7f8a48100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 -- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 msgr2=0x7f8a48100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 -- 172.21.15.96:0/434145433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a48100fe0 con 0x7f8a48102260 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800e9f0 tx=0x7f8a3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a44d99640 1 -- 172.21.15.96:0/434145433 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800ccc0 con 0x7f8a48102260 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a44d99640 1 -- 172.21.15.96:0/434145433 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a38005020 con 0x7f8a48102260 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a481a1c60 con 0x7f8a48102260 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a44d99640 1 -- 172.21.15.96:0/434145433 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a380049e0 con 0x7f8a48102260 2024-03-18T15:42:54.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.025+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481a2130 con 0x7f8a48102260 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a44d99640 1 -- 172.21.15.96:0/434145433 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f8a38005560 con 0x7f8a48102260 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 Processor -- start 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 -- start start 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c6850 0x55c4de8c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c7240 0x55c4de8cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c4de82b490 con 0x55c4de8c6850 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c4de7d3c90 con 0x55c4de8c7240 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4de827530 con 0x55c4de8cc3b0 2024-03-18T15:42:54.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a44d99640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 0x7f8a340a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.028+0000 7f8a4759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c6850 0x55c4de8c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a46d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c7240 0x55c4de8cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:59790/0 (socket says 172.21.15.96:59790) 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 -- 172.21.15.96:0/2121291719 learned_addr learned my addr 172.21.15.96:0/2121291719 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a4759e640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 0x7f8a340a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a44d99640 1 -- 172.21.15.96:0/434145433 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8a380ceab0 con 0x7f8a48102260 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 -- 172.21.15.96:0/2121291719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c7240 msgr2=0x55c4de8cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c7240 0x55c4de8cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 -- 172.21.15.96:0/2121291719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c6850 msgr2=0x55c4de8c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c6850 0x55c4de8c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 -- 172.21.15.96:0/2121291719 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4de88e500 con 0x55c4de8cc3b0 2024-03-18T15:42:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a4759e640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 0x7f8a340a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55c4de8c7aa0 tx=0x7f8a40009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000c00 tx=0x7f8a3002cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71dd70df3ac6bc26 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3d7fa640 1 -- 172.21.15.96:0/2121291719 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30042070 con 0x55c4de8cc3b0 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3d7fa640 1 -- 172.21.15.96:0/2121291719 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30004ba0 con 0x55c4de8cc3b0 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.029+0000 7f8a3d7fa640 1 -- 172.21.15.96:0/2121291719 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30005240 con 0x55c4de8cc3b0 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/2121291719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 msgr2=0x55c4de8ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000c00 tx=0x7f8a3002cd30 comp rx=0 tx=0).stop 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/2121291719 shutdown_connections 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8ce840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c7240 0x55c4de8cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/2121291719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c6850 0x55c4de8c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/2121291719 >> 172.21.15.96:0/2121291719 conn(0x55c4de8c1dc0 msgr2=0x55c4de8c21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/2121291719 shutdown_connections 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/2121291719 wait complete. 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 -- start start 2024-03-18T15:42:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.030+0000 7f8a4fc3b740 1 Processor -- start 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c4de8cf300 con 0x55c4de8c7240 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c4de7e7230 con 0x55c4de8c6850 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4de8cf480 con 0x55c4de8cc3b0 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 10 client.0 _ll_register_callbacks cb 0x55c4de823010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56320/0 (socket says 172.21.15.96:56320) 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 -- 172.21.15.96:0/3889967100 learned_addr learned my addr 172.21.15.96:0/3889967100 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4759e640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 -- 172.21.15.96:0/3889967100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 msgr2=0x55c4de8d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 -- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 msgr2=0x55c4de8d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 -- 172.21.15.96:0/3889967100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4de7f6100 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f624791c5088665b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3004e060 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8a30034440 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.031+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a30004ba0 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a30042070 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4de53f3f0 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a30004ba0 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f8a30054530 con 0x55c4de8c7240 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 10 client.16302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:42:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 20 client.16302 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de8dc7d0 0x55c4de8debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 4 client.16302 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4de8389d0 con 0x55c4de8dc7d0 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.032+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de8dc7d0 0x55c4de8debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.033+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de8dc7d0 0x55c4de8debc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4a34f6a88dbdee3 server_cookie=63dce09ee4cbb7fe in_seq=0 out_seq=0 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.033+0000 7f8a2e7fc640 10 client.16302 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.033+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/3889967100 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c4de8389d0 con 0x55c4de8dc7d0 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.033+0000 7f8a2e7fc640 10 client.16302 handle_command_reply: tid=0 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 client.16302 shutdown 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 msgr2=0x55c4de8d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006010 tx=0x7f8a30004dc0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 shutdown_connections 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d1330 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de8dc7d0 0x55c4de8debc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/3889967100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a2e7fc640 0 client.16302 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 >> 172.21.15.96:0/3889967100 conn(0x55c4de8c1dc0 msgr2=0x55c4de8c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 shutdown_connections 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/3889967100 wait complete. 2024-03-18T15:42:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.034+0000 7f8a4fc3b740 20 client.16302 trim_cache size 0 max 16384 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.048+0000 7f8a4fc3b740 1 Processor -- start 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.048+0000 7f8a4fc3b740 1 -- start start 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.048+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.048+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481aaef0 con 0x55c4de8c7240 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481acb50 con 0x55c4de8c6850 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481ab5c0 con 0x55c4de8cc3b0 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56328/0 (socket says 172.21.15.96:56328) 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 -- 172.21.15.96:0/1838922841 learned_addr learned my addr 172.21.15.96:0/1838922841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a4759e640 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 -- 172.21.15.96:0/1838922841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 msgr2=0x55c4de8e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 -- 172.21.15.96:0/1838922841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 msgr2=0x55c4de8c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 -- 172.21.15.96:0/1838922841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4de8bd640 con 0x55c4de8c7240 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8a30054b40 tx=0x7f8a3002cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef553929cb0e080 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3cff9640 1 -- 172.21.15.96:0/1838922841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30005d50 con 0x55c4de8c7240 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3cff9640 1 -- 172.21.15.96:0/1838922841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a300049d0 con 0x55c4de8c7240 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.049+0000 7f8a3cff9640 1 -- 172.21.15.96:0/1838922841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300346a0 con 0x55c4de8c7240 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/1838922841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 msgr2=0x55c4de8d8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8a30054b40 tx=0x7f8a3002cd30 comp rx=0 tx=0).stop 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/1838922841 shutdown_connections 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d8280 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/1838922841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/1838922841 >> 172.21.15.96:0/1838922841 conn(0x55c4de8c1dc0 msgr2=0x55c4de8c5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/1838922841 shutdown_connections 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/1838922841 wait complete. 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 -- start start 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.050+0000 7f8a4fc3b740 1 Processor -- start 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481ac3c0 con 0x55c4de8c7240 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481abcf0 con 0x55c4de8c6850 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a481a8ed0 con 0x55c4de8cc3b0 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 10 client.0 _ll_register_callbacks cb 0x7f8a380046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56338/0 (socket says 172.21.15.96:56338) 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 -- 172.21.15.96:0/926705834 learned_addr learned my addr 172.21.15.96:0/926705834 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4759e640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 -- 172.21.15.96:0/926705834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 msgr2=0x55c4de8d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 -- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 msgr2=0x55c4de8d5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 -- 172.21.15.96:0/926705834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4de6097d0 con 0x55c4de8c7240 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a3ffff640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8a30004620 tx=0x7f8a3002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aad7f651e5937dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3004e040 con 0x55c4de8c7240 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f8a30034440 con 0x55c4de8c7240 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a300049d0 con 0x55c4de8c7240 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a3006e050 con 0x55c4de8c7240 2024-03-18T15:42:54.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 10 client.16308 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.051+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4de8d83a0 con 0x55c4de8c7240 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a300049d0 con 0x55c4de8c7240 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7f8a3005d430 con 0x55c4de8c7240 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 10 client.16308 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 10 client.16308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 20 client.16308 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de942ee0 0x55c4de945300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 4 client.16308 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4de832d20 con 0x55c4de942ee0 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de942ee0 0x55c4de945300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a46d9d640 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de942ee0 0x55c4de945300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c18a3397aa65c10 server_cookie=cbb2898f3e0351c4 in_seq=0 out_seq=0 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.052+0000 7f8a2e7fc640 10 client.16308 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a2e7fc640 1 -- 172.21.15.96:0/926705834 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4de832d20 con 0x55c4de942ee0 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a2e7fc640 10 client.16308 handle_command_reply: tid=0 2024-03-18T15:42:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 client.16308 shutdown 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 msgr2=0x55c4de8d5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8a30004620 tx=0x7f8a3002bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 shutdown_connections 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c4de8cc3b0 0x55c4de8d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4de8c7240 0x55c4de8d5c40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55c4de942ee0 0x55c4de945300 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 --2- 172.21.15.96:0/926705834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c4de8c6850 0x55c4de8d5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a2e7fc640 0 client.16308 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 >> 172.21.15.96:0/926705834 conn(0x55c4de8c1dc0 msgr2=0x55c4de8c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 shutdown_connections 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 1 -- 172.21.15.96:0/926705834 wait complete. 2024-03-18T15:42:54.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4fc3b740 20 client.16308 trim_cache size 0 max 16384 2024-03-18T15:42:54.061 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:42:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 msgr2=0x7f8a340a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 0x7f8a340a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x55c4de8c7aa0 tx=0x7f8a40009500 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 msgr2=0x7f8a48101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800e9f0 tx=0x7f8a3800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 shutdown_connections 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f8a340a2800 0x7f8a340a4cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a481074c0 0x7f8a48100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a48104b90 0x7f8a48100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 --2- 172.21.15.96:0/434145433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a48102260 0x7f8a48101bd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 >> 172.21.15.96:0/434145433 conn(0x7f8a480fbc70 msgr2=0x7f8a48102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 shutdown_connections 2024-03-18T15:42:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.053+0000 7f8a4d04b640 1 -- 172.21.15.96:0/434145433 wait complete. 2024-03-18T15:42:54.065 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:42:54.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 -- 172.21.15.96:0/4267578605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 msgr2=0x7fb62c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 --2- 172.21.15.96:0/4267578605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c100df0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 -- 172.21.15.96:0/4267578605 shutdown_connections 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 --2- 172.21.15.96:0/4267578605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101330 0x7fb62c108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 --2- 172.21.15.96:0/4267578605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c100df0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 --2- 172.21.15.96:0/4267578605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101d20 0x7fb62c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 -- 172.21.15.96:0/4267578605 >> 172.21.15.96:0/4267578605 conn(0x7fb62c0fb3e0 msgr2=0x7fb62c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 -- 172.21.15.96:0/4267578605 shutdown_connections 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.174+0000 7fb633b09640 1 -- 172.21.15.96:0/4267578605 wait complete. 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 Processor -- start 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 -- start start 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101330 0x7fb62c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101d20 0x7fb62c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c3b0 con 0x7fb62c100970 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c230 con 0x7fb62c101330 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb633b09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c530 con 0x7fb62c101d20 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb6318a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb6318a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56354/0 (socket says 172.21.15.96:56354) 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb6318a5640 1 -- 172.21.15.96:0/1790186118 learned_addr learned my addr 172.21.15.96:0/1790186118 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb6310a4640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101330 0x7fb62c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.175+0000 7fb6320a6640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101d20 0x7fb62c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 -- 172.21.15.96:0/1790186118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101d20 msgr2=0x7fb62c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101d20 0x7fb62c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 -- 172.21.15.96:0/1790186118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101330 msgr2=0x7fb62c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101330 0x7fb62c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 -- 172.21.15.96:0/1790186118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c19f740 con 0x7fb62c100970 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb6318a5640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb62800e9c0 tx=0x7fb62800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb622ffd640 1 -- 172.21.15.96:0/1790186118 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800cde0 con 0x7fb62c100970 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c19fa30 con 0x7fb62c100970 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb622ffd640 1 -- 172.21.15.96:0/1790186118 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb628010910 con 0x7fb62c100970 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb622ffd640 1 -- 172.21.15.96:0/1790186118 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628018a10 con 0x7fb62c100970 2024-03-18T15:42:54.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.176+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c19ffa0 con 0x7fb62c100970 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 Processor -- start 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb622ffd640 1 -- 172.21.15.96:0/1790186118 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb62801f020 con 0x7fb62c100970 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 -- start start 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf0240 0x55965dbf4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55965db54490 con 0x55965dbf53b0 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55965dafcc90 con 0x55965dbef850 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.179+0000 7fb6367d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55965db50530 con 0x55965dbf0240 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6310a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:60096/0 (socket says 172.21.15.96:60096) 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 -- 172.21.15.96:0/2583345810 learned_addr learned my addr 172.21.15.96:0/2583345810 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6318a5640 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf0240 0x55965dbf4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 -- 172.21.15.96:0/2583345810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf0240 msgr2=0x55965dbf4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf0240 0x55965dbf4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 -- 172.21.15.96:0/2583345810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 msgr2=0x55965dbf7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 -- 172.21.15.96:0/2583345810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965dbb7500 con 0x55965dbef850 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb622ffd640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 0x7fb6180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb624009fd0 tx=0x7fb6240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e7fc099107e54f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb6318a5640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 0x7fb6180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb622ffd640 1 -- 172.21.15.96:0/1790186118 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb6280cec50 con 0x7fb62c100970 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb617fff640 1 -- 172.21.15.96:0/2583345810 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624004d20 con 0x55965dbef850 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb617fff640 1 -- 172.21.15.96:0/2583345810 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624004050 con 0x55965dbef850 2024-03-18T15:42:54.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.180+0000 7fb617fff640 1 -- 172.21.15.96:0/2583345810 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624005260 con 0x55965dbef850 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- 172.21.15.96:0/2583345810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 msgr2=0x55965dbefc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6318a5640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 0x7fb6180a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6280cca90 tx=0x7fb6280d2730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb624009fd0 tx=0x7fb6240118d0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- 172.21.15.96:0/2583345810 shutdown_connections 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf0240 0x55965dbf4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2583345810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dbefc70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- 172.21.15.96:0/2583345810 >> 172.21.15.96:0/2583345810 conn(0x55965dbeadc0 msgr2=0x55965dbeb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- 172.21.15.96:0/2583345810 shutdown_connections 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- 172.21.15.96:0/2583345810 wait complete. 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 -- start start 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.181+0000 7fb6367d3740 1 Processor -- start 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbf9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dbfa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dc01240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55965dbf8360 con 0x55965dbf53b0 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55965db10230 con 0x55965dbf0240 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55965dbf84e0 con 0x55965dbef850 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6367d3740 10 client.0 _ll_register_callbacks cb 0x55965db4c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dc01240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbf9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbf9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:59856/0 (socket says 172.21.15.96:59856) 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 -- 172.21.15.96:0/1658199389 learned_addr learned my addr 172.21.15.96:0/1658199389 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6310a4640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dbfa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 -- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 msgr2=0x55965dbf9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbf9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 -- 172.21.15.96:0/1658199389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 msgr2=0x55965dbfa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dbfa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 -- 172.21.15.96:0/1658199389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965db1f100 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6320a6640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dc01240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb624004e60 tx=0x7fb624009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed0587d25823e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.182+0000 7fb6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62402d060 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb62400f4e0 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb624004050 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb624004690 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55965d8683f0 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624004050 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fb624025040 con 0x55965dbf53b0 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 20 client.16320 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc05940 0x55965dc07d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 4 client.16320 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55965db619d0 con 0x55965dc05940 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.183+0000 7fb6310a4640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc05940 0x55965dc07d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.184+0000 7fb6157fa640 10 client.16320 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.184+0000 7fb6310a4640 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc05940 0x55965dc07d70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2916ef661d2312ee server_cookie=368cb56b45832ed4 in_seq=0 out_seq=0 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6157fa640 1 -- 172.21.15.96:0/1658199389 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55965db619d0 con 0x55965dc05940 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6157fa640 10 client.16320 handle_command_reply: tid=0 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 client.16320 shutdown 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 msgr2=0x55965dc01240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dc01240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb624004e60 tx=0x7fb624009500 comp rx=0 tx=0).stop 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 shutdown_connections 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc05940 0x55965dc07d70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dc01240 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dbfa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 --2- 172.21.15.96:0/1658199389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbf9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6157fa640 0 client.16320 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 >> 172.21.15.96:0/1658199389 conn(0x55965dbeadc0 msgr2=0x55965dbe9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 shutdown_connections 2024-03-18T15:42:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 1 -- 172.21.15.96:0/1658199389 wait complete. 2024-03-18T15:42:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.185+0000 7fb6367d3740 20 client.16320 trim_cache size 0 max 16384 2024-03-18T15:42:54.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 Processor -- start 2024-03-18T15:42:54.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 -- start start 2024-03-18T15:42:54.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbe9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dc01110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1a8e40 con 0x55965dbf53b0 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1aaaa0 con 0x55965dbf0240 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6367d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1a9510 con 0x55965dbef850 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6320a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56376/0 (socket says 172.21.15.96:56376) 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6320a6640 1 -- 172.21.15.96:0/3641353456 learned_addr learned my addr 172.21.15.96:0/3641353456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6310a4640 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dc01110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.199+0000 7fb6320a6640 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbe9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 -- 172.21.15.96:0/3641353456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 msgr2=0x55965dbe9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbe9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 -- 172.21.15.96:0/3641353456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 msgr2=0x55965dc01110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dc01110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 -- 172.21.15.96:0/3641353456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965dbe6640 con 0x55965dbf53b0 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6320a6640 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb624003e70 tx=0x7fb6240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07604df66a0d83 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6177fe640 1 -- 172.21.15.96:0/3641353456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624009ea0 con 0x55965dbf53b0 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6177fe640 1 -- 172.21.15.96:0/3641353456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624004690 con 0x55965dbf53b0 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.200+0000 7fb6177fe640 1 -- 172.21.15.96:0/3641353456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624004ab0 con 0x55965dbf53b0 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- 172.21.15.96:0/3641353456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 msgr2=0x55965dbf8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb624003e70 tx=0x7fb6240118d0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- 172.21.15.96:0/3641353456 shutdown_connections 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf53b0 0x55965dbf8880 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbf0240 0x55965dc01110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- 172.21.15.96:0/3641353456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbef850 0x55965dbe9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- 172.21.15.96:0/3641353456 >> 172.21.15.96:0/3641353456 conn(0x55965dbeadc0 msgr2=0x55965dbe9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- 172.21.15.96:0/3641353456 shutdown_connections 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- 172.21.15.96:0/3641353456 wait complete. 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- start start 2024-03-18T15:42:54.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 Processor -- start 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dc00fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf53b0 0x55965dbfdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1aa310 con 0x55965dbf0240 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1a9c40 con 0x55965dbef850 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1a6e20 con 0x55965dbf53b0 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.201+0000 7fb6367d3740 10 client.0 _ll_register_callbacks cb 0x7fb6240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56388/0 (socket says 172.21.15.96:56388) 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 -- 172.21.15.96:0/2699800537 learned_addr learned my addr 172.21.15.96:0/2699800537 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dc00fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf53b0 0x55965dbfdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 -- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf53b0 msgr2=0x55965dbfdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf53b0 0x55965dbfdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 -- 172.21.15.96:0/2699800537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 msgr2=0x55965dc00fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dc00fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 -- 172.21.15.96:0/2699800537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965d9327d0 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6310a4640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02c7b0 tx=0x7fb61c02cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f0092eac1819ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c02cd10 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb61c046390 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb624004690 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb61c054030 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6367d3740 10 client.16326 fetch_fsmap learned FSMap version 29 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55965dc6add0 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624004690 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.202+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8378+0+0 (secure 0 0 0) 0x7fb61c046070 con 0x55965dbf0240 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 10 client.16326 fetch_fsmap finished waiting for FSMap version 29 2024-03-18T15:42:54.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 20 client.16326 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc6b870 0x55965dc6dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 4 client.16326 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55965db5bd20 con 0x55965dc6b870 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc6b870 0x55965dc6dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6320a6640 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc6b870 0x55965dc6dc90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806981221b960917 server_cookie=ac0ab96737fc0a6f in_seq=0 out_seq=0 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6157fa640 10 client.16326 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6157fa640 1 -- 172.21.15.96:0/2699800537 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55965db5bd20 con 0x55965dc6b870 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6157fa640 10 client.16326 handle_command_reply: tid=0 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 1 client.16326 shutdown 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 msgr2=0x55965dbfda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.203+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02c7b0 tx=0x7fb61c02cbe0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 shutdown_connections 2024-03-18T15:42:54.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55965dbf53b0 0x55965dbfdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55965dc6b870 0x55965dc6dc90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55965dbf0240 0x55965dbfda50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 --2- 172.21.15.96:0/2699800537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55965dbef850 0x55965dc00fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6157fa640 0 client.16326 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 >> 172.21.15.96:0/2699800537 conn(0x55965dbeadc0 msgr2=0x55965dbee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 shutdown_connections 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 1 -- 172.21.15.96:0/2699800537 wait complete. 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb6367d3740 20 client.16326 trim_cache size 0 max 16384 2024-03-18T15:42:54.211 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 msgr2=0x7fb6180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 0x7fb6180a4cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb6280cca90 tx=0x7fb6280d2730 comp rx=0 tx=0).stop 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 msgr2=0x7fb62c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb62800e9c0 tx=0x7fb62800ee80 comp rx=0 tx=0).stop 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 shutdown_connections 2024-03-18T15:42:54.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb6180a2800 0x7fb6180a4cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb62c101d20 0x7fb62c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb62c101330 0x7fb62c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 --2- 172.21.15.96:0/1790186118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb62c100970 0x7fb62c197b20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 >> 172.21.15.96:0/1790186118 conn(0x7fb62c0fb3e0 msgr2=0x7fb62c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 shutdown_connections 2024-03-18T15:42:54.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:54.204+0000 7fb633b09640 1 -- 172.21.15.96:0/1790186118 wait complete. 2024-03-18T15:42:54.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:54 smithi137 ceph-mon[35868]: pgmap v436: 97 pgs: 97 active+clean; 305 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-18T15:42:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:54 smithi193 ceph-mon[36702]: pgmap v436: 97 pgs: 97 active+clean; 305 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-18T15:42:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:54 smithi096 ceph-mon[31930]: pgmap v436: 97 pgs: 97 active+clean; 305 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-03-18T15:42:55.217 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:42:55.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:42:55.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2339862320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54100720 msgr2=0x7f5d54100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2339862320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54100720 0x7f5d54100b40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5d50009900 tx=0x7f5d5002bc30 comp rx=0 tx=0).stop 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2339862320 shutdown_connections 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2339862320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d541022d0 0x7f5d54108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2339862320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d54101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2339862320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d54100720 0x7f5d54100b40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2339862320 >> 172.21.15.96:0/2339862320 conn(0x7f5d540fbc90 msgr2=0x7f5d540fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2339862320 shutdown_connections 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.331+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2339862320 wait complete. 2024-03-18T15:42:55.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 Processor -- start 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 -- start start 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d54100720 0x7f5d5406a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d5419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410c3f0 con 0x7f5d541022d0 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410c270 con 0x7f5d54101910 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d5b4dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410c570 con 0x7f5d54100720 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d59a79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d5419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d59a79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56414/0 (socket says 172.21.15.96:56414) 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d59a79640 1 -- 172.21.15.96:0/2423121935 learned_addr learned my addr 172.21.15.96:0/2423121935 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d59278640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d54100720 0x7f5d5406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 -- 172.21.15.96:0/2423121935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d54100720 msgr2=0x7f5d5406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d54100720 0x7f5d5406a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 -- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 msgr2=0x7f5d5419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 -- 172.21.15.96:0/2423121935 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5419f840 con 0x7f5d54101910 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.332+0000 7f5d58a77640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d5419c910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e9f0 tx=0x7f5d4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.333+0000 7f5d4a7fc640 1 -- 172.21.15.96:0/2423121935 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400ccc0 con 0x7f5d54101910 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.333+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5419fad0 con 0x7f5d54101910 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.333+0000 7f5d4a7fc640 1 -- 172.21.15.96:0/2423121935 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d44010910 con 0x7f5d54101910 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.333+0000 7f5d4a7fc640 1 -- 172.21.15.96:0/2423121935 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44018960 con 0x7f5d54101910 2024-03-18T15:42:55.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.333+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d541a0070 con 0x7f5d54101910 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.335+0000 7f5d4a7fc640 1 -- 172.21.15.96:0/2423121935 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5d44010ac0 con 0x7f5d54101910 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.335+0000 7f5d5e1a1740 1 Processor -- start 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.335+0000 7f5d5e1a1740 1 -- start start 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.335+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b59db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b5ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b61980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5abe0d0 con 0x55c0f5b5a380 2024-03-18T15:42:55.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5a66c70 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5aba170 con 0x55c0f5b59990 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d4a7fc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 0x7f5d400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b5ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b59db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d58a77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b61980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b5ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56428/0 (socket says 172.21.15.96:56428) 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59278640 1 -- 172.21.15.96:0/2097707013 learned_addr learned my addr 172.21.15.96:0/2097707013 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 0x7f5d400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d4a7fc640 1 -- 172.21.15.96:0/2423121935 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d440cee40 con 0x7f5d54101910 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 -- 172.21.15.96:0/2097707013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 msgr2=0x55c0f5b61980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b61980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 -- 172.21.15.96:0/2097707013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 msgr2=0x55c0f5b5ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b5ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 -- 172.21.15.96:0/2097707013 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f589a7e0 con 0x55c0f5b59990 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 0x7f5d400a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e350 tx=0x7f5d4c018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d59a79640 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b59db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c011620 tx=0x7f5d4c011ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3612d3ff45b2e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d3f7fe640 1 -- 172.21.15.96:0/2097707013 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c025070 con 0x55c0f5b59990 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d3f7fe640 1 -- 172.21.15.96:0/2097707013 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d4c0136b0 con 0x55c0f5b59990 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.336+0000 7f5d3f7fe640 1 -- 172.21.15.96:0/2097707013 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c0055f0 con 0x55c0f5b59990 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/2097707013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 msgr2=0x55c0f5b59db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b59db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c011620 tx=0x7f5d4c011ae0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/2097707013 shutdown_connections 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b61980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b5ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/2097707013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b59db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/2097707013 >> 172.21.15.96:0/2097707013 conn(0x55c0f5b21c50 msgr2=0x55c0f5b55300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/2097707013 shutdown_connections 2024-03-18T15:42:55.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.337+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/2097707013 wait complete. 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 -- start start 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 Processor -- start 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b63e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5b62440 con 0x55c0f5b5a380 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5a7a0d0 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0f5b625c0 con 0x55c0f5b59990 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d5e1a1740 10 client.0 _ll_register_callbacks cb 0x55c0f5a43760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d59278640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b63e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56430/0 (socket says 172.21.15.96:56430) 2024-03-18T15:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 -- 172.21.15.96:0/1825728849 learned_addr learned my addr 172.21.15.96:0/1825728849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d59a79640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 -- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 msgr2=0x55c0f5b63e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b63e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 -- 172.21.15.96:0/1825728849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 msgr2=0x55c0f5b6b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.338+0000 7f5d58a77640 1 -- 172.21.15.96:0/1825728849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f58faf70 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d58a77640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d4409b560 tx=0x7f5d44066620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beed6f2fb08a84e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44026040 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d440860b0 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d4c0136b0 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d4409b1c0 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 10 client.16338 fetch_fsmap learned FSMap version 30 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0f5792170 con 0x55c0f5b5a380 2024-03-18T15:42:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d4c0136b0 con 0x55c0f5b5a380 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5d4406a250 con 0x55c0f5b5a380 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 10 client.16338 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 20 client.16338 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5b6f7d0 0x55c0f5b71c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 4 client.16338 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.339+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f5acb610 con 0x55c0f5b6f7d0 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.340+0000 7f5d59278640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5b6f7d0 0x55c0f5b71c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.340+0000 7f5d3cff9640 10 client.16338 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.340+0000 7f5d59278640 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5b6f7d0 0x55c0f5b71c10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42f9f1a9431e6e3 server_cookie=f726fa63e3695c8f in_seq=0 out_seq=0 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.340+0000 7f5d3cff9640 1 -- 172.21.15.96:0/1825728849 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c0f5acb610 con 0x55c0f5b6f7d0 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.340+0000 7f5d3cff9640 10 client.16338 handle_command_reply: tid=0 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 client.16338 shutdown 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 msgr2=0x55c0f5b64360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5d4409b560 tx=0x7f5d44066620 comp rx=0 tx=0).stop 2024-03-18T15:42:55.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 shutdown_connections 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5b6f7d0 0x55c0f5b71c10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5a380 0x55c0f5b64360 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1825728849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b59990 0x55c0f5b63e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d3cff9640 0 client.16338 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 >> 172.21.15.96:0/1825728849 conn(0x55c0f5b21c50 msgr2=0x55c0f5b544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 shutdown_connections 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1825728849 wait complete. 2024-03-18T15:42:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.341+0000 7f5d5e1a1740 20 client.16338 trim_cache size 0 max 16384 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 Processor -- start 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 -- start start 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b6b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b58a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a8f10 con 0x55c0f5b59990 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541aaae0 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.358+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a95e0 con 0x55c0f5b5a380 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56440/0 (socket says 172.21.15.96:56440) 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 -- 172.21.15.96:0/1408277340 learned_addr learned my addr 172.21.15.96:0/1408277340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d58a77640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b6b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59a79640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b58a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 -- 172.21.15.96:0/1408277340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 msgr2=0x55c0f5b6b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b6b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 -- 172.21.15.96:0/1408277340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 msgr2=0x55c0f5b58a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b58a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 -- 172.21.15.96:0/1408277340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f5b558a0 con 0x55c0f5b59990 2024-03-18T15:42:55.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59a79640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d59278640 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5d50036040 tx=0x7f5d50005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea670ff2fef7b7d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d3effd640 1 -- 172.21.15.96:0/1408277340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5004dba0 con 0x55c0f5b59990 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d3effd640 1 -- 172.21.15.96:0/1408277340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5004dd40 con 0x55c0f5b59990 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.359+0000 7f5d3effd640 1 -- 172.21.15.96:0/1408277340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d500046c0 con 0x55c0f5b59990 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1408277340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 msgr2=0x55c0f5b58540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5d50036040 tx=0x7f5d50005c00 comp rx=0 tx=0).stop 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1408277340 shutdown_connections 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b58a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b6b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/1408277340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b59990 0x55c0f5b58540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1408277340 >> 172.21.15.96:0/1408277340 conn(0x55c0f5b21c50 msgr2=0x55c0f5b63e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1408277340 shutdown_connections 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/1408277340 wait complete. 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 -- start start 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.360+0000 7f5d5e1a1740 1 Processor -- start 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 0x55c0f5b69130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b69670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541aa3e0 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a9d10 con 0x55c0f5b59990 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d541a6ef0 con 0x55c0f5b5a380 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 10 client.0 _ll_register_callbacks cb 0x7f5d4400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56450/0 (socket says 172.21.15.96:56450) 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 -- 172.21.15.96:0/4121033611 learned_addr learned my addr 172.21.15.96:0/4121033611 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d58a77640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b69670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59278640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 0x55c0f5b69130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 -- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 msgr2=0x55c0f5b69670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b69670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 -- 172.21.15.96:0/4121033611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 msgr2=0x55c0f5b69130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 0x55c0f5b69130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 -- 172.21.15.96:0/4121033611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f5b6ac70 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59278640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 0x55c0f5b69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d59a79640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55c0f5b5abe0 tx=0x7f5d4c013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3c540e72767694 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c01c040 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5d4c021840 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d5004dd40 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5d4c025040 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.361+0000 7f5d5e1a1740 10 client.16344 fetch_fsmap learned FSMap version 30 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0f5bd6230 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d5004dd40 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5d4c03b030 con 0x55c0f5b5f4f0 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 10 client.16344 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:55.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 20 client.16344 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5bd6ad0 0x55c0f5bd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 4 client.16344 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0f5b63c10 con 0x55c0f5bd6ad0 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.362+0000 7f5d59278640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5bd6ad0 0x55c0f5bd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.363+0000 7f5d3cff9640 10 client.16344 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.363+0000 7f5d59278640 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5bd6ad0 0x55c0f5bd8ef0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2544301c2a54e59e server_cookie=7c7388b3985b67ef in_seq=0 out_seq=0 2024-03-18T15:42:55.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d3cff9640 1 -- 172.21.15.96:0/4121033611 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0f5b63c10 con 0x55c0f5bd6ad0 2024-03-18T15:42:55.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d3cff9640 10 client.16344 handle_command_reply: tid=0 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 client.16344 shutdown 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 msgr2=0x55c0f5b6a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55c0f5b5abe0 tx=0x7f5d4c013e20 comp rx=0 tx=0).stop 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 shutdown_connections 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c0f5bd6ad0 0x55c0f5bd8ef0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0f5b5f4f0 0x55c0f5b6a680 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0f5b5a380 0x55c0f5b69670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 --2- 172.21.15.96:0/4121033611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0f5b59990 0x55c0f5b69130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 >> 172.21.15.96:0/4121033611 conn(0x55c0f5b21c50 msgr2=0x55c0f5b764d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d3cff9640 0 client.16344 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 shutdown_connections 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 1 -- 172.21.15.96:0/4121033611 wait complete. 2024-03-18T15:42:55.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.364+0000 7f5d5e1a1740 20 client.16344 trim_cache size 0 max 16384 2024-03-18T15:42:55.373 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:42:55.373 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:42:55.373 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "a795e0da-d125-4831-b835-6249275c175d", 2024-03-18T15:42:55.373 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:42:55.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:42:55.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 msgr2=0x7f5d400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 0x7f5d400a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00e350 tx=0x7f5d4c018000 comp rx=0 tx=0).stop 2024-03-18T15:42:55.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 msgr2=0x7f5d5419c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d5419c910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400e9f0 tx=0x7f5d4400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 shutdown_connections 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5d400a2800 0x7f5d400a4cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d541022d0 0x7f5d5419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d54101910 0x7f5d5419c910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 --2- 172.21.15.96:0/2423121935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d54100720 0x7f5d5406a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 >> 172.21.15.96:0/2423121935 conn(0x7f5d540fbc90 msgr2=0x7f5d54104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 shutdown_connections 2024-03-18T15:42:55.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.365+0000 7f5d5b4dc640 1 -- 172.21.15.96:0/2423121935 wait complete. 2024-03-18T15:42:55.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:42:55.381 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:42:55.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- 172.21.15.96:0/1145066759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc101990 msgr2=0x7f60bc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 --2- 172.21.15.96:0/1145066759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc101990 0x7f60bc1082b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009ca0 tx=0x7f60b402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- 172.21.15.96:0/1145066759 shutdown_connections 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 --2- 172.21.15.96:0/1145066759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc101990 0x7f60bc1082b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 --2- 172.21.15.96:0/1145066759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc100fd0 0x7f60bc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 --2- 172.21.15.96:0/1145066759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc0ffe80 0x7f60bc1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- 172.21.15.96:0/1145066759 >> 172.21.15.96:0/1145066759 conn(0x7f60bc0fb410 msgr2=0x7f60bc0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- 172.21.15.96:0/1145066759 shutdown_connections 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- 172.21.15.96:0/1145066759 wait complete. 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 Processor -- start 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.498+0000 7f60c3080640 1 -- start start 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc100fd0 0x7f60bc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc101990 0x7f60bc19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10b2a0 con 0x7f60bc0ffe80 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10b120 con 0x7f60bc101990 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc10b420 con 0x7f60bc100fd0 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56460/0 (socket says 172.21.15.96:56460) 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 -- 172.21.15.96:0/3893311072 learned_addr learned my addr 172.21.15.96:0/3893311072 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c161d640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc101990 0x7f60bc19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60bbfff640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc100fd0 0x7f60bc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 -- 172.21.15.96:0/3893311072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc100fd0 msgr2=0x7f60bc198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc100fd0 0x7f60bc198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 -- 172.21.15.96:0/3893311072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc101990 msgr2=0x7f60bc19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc101990 0x7f60bc19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 -- 172.21.15.96:0/3893311072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc19f6f0 con 0x7f60bc0ffe80 2024-03-18T15:42:55.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60b000dbc0 tx=0x7f60b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60b9ffb640 1 -- 172.21.15.96:0/3893311072 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0005220 con 0x7f60bc0ffe80 2024-03-18T15:42:55.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60b9ffb640 1 -- 172.21.15.96:0/3893311072 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b000ae50 con 0x7f60bc0ffe80 2024-03-18T15:42:55.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60b9ffb640 1 -- 172.21.15.96:0/3893311072 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000e870 con 0x7f60bc0ffe80 2024-03-18T15:42:55.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.499+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc19f9e0 con 0x7f60bc0ffe80 2024-03-18T15:42:55.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.500+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc19ff30 con 0x7f60bc0ffe80 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.502+0000 7f60b9ffb640 1 -- 172.21.15.96:0/3893311072 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f60b000ea10 con 0x7f60bc0ffe80 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.502+0000 7f60c5c5f740 1 Processor -- start 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 -- start start 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 0x557999631590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x557999636660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 0x557999639160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557999596500 con 0x557999631170 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55799953ebe0 con 0x557999636cd0 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60b9ffb640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 0x7f60ac0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5579995925a0 con 0x557999631b60 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 0x557999631590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 0x557999639160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 0x557999631590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56474/0 (socket says 172.21.15.96:56474) 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x557999636660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c161d640 1 -- 172.21.15.96:0/4195268856 learned_addr learned my addr 172.21.15.96:0/4195268856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c161d640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 0x7f60ac0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60b9ffb640 1 -- 172.21.15.96:0/3893311072 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f60b00cea60 con 0x7f60bc0ffe80 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 -- 172.21.15.96:0/4195268856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 msgr2=0x557999636660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x557999636660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 -- 172.21.15.96:0/4195268856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 msgr2=0x557999631590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 0x557999631590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 -- 172.21.15.96:0/4195268856 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579995524b0 con 0x557999636cd0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60c161d640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 0x7f60ac0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5579996323c0 tx=0x7f60b4032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.503+0000 7f60bbfff640 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 0x557999639160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0ffba0 tx=0x7f60a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90f66aae000be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60a6ffd640 1 -- 172.21.15.96:0/4195268856 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8009d70 con 0x557999636cd0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60a6ffd640 1 -- 172.21.15.96:0/4195268856 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a8005020 con 0x557999636cd0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60a6ffd640 1 -- 172.21.15.96:0/4195268856 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8004ab0 con 0x557999636cd0 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4195268856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 msgr2=0x557999639160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 0x557999639160 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f60bc0ffba0 tx=0x7f60a8011810 comp rx=0 tx=0).stop 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4195268856 shutdown_connections 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999636cd0 0x557999639160 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x557999636660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4195268856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999631170 0x557999631590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4195268856 >> 172.21.15.96:0/4195268856 conn(0x55799962c6e0 msgr2=0x55799962cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4195268856 shutdown_connections 2024-03-18T15:42:55.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.504+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4195268856 wait complete. 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 -- start start 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 Processor -- start 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x55799963b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x55799963bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557999639c80 con 0x557999636cd0 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557999552160 con 0x557999631170 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557999639e00 con 0x557999631b60 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c5c5f740 10 client.0 _ll_register_callbacks cb 0x557999536150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x55799963bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56482/0 (socket says 172.21.15.96:56482) 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 -- 172.21.15.96:0/2836847186 learned_addr learned my addr 172.21.15.96:0/2836847186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x55799963b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 -- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 msgr2=0x55799963bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x55799963bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 -- 172.21.15.96:0/2836847186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 msgr2=0x55799963b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.505+0000 7f60c161d640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x55799963b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c161d640 1 -- 172.21.15.96:0/2836847186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579993747d0 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c161d640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c700 tx=0x7f60b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a15cd72726fb97a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4054060 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f60b4004250 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a8005020 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60b4039040 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 10 client.16356 fetch_fsmap learned FSMap version 30 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579992aa3f0 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a8005020 con 0x557999636cd0 2024-03-18T15:42:55.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f60b404f470 con 0x557999636cd0 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 10 client.16356 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 20 client.16356 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 4 client.16356 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.506+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579995a3a40 con 0x557999647130 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.507+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.507+0000 7f609bfff640 10 client.16356 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.507+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996495c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474c05b982e8e3da server_cookie=66483c7487d2c856 in_seq=0 out_seq=0 2024-03-18T15:42:55.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f609bfff640 1 -- 172.21.15.96:0/2836847186 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5579995a3a40 con 0x557999647130 2024-03-18T15:42:55.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f609bfff640 10 client.16356 handle_command_reply: tid=0 2024-03-18T15:42:55.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 client.16356 shutdown 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 msgr2=0x557999642b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c700 tx=0x7f60b40095c0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 shutdown_connections 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996495c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f609bfff640 0 client.16356 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999642b40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x55799963bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/2836847186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x55799963b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 >> 172.21.15.96:0/2836847186 conn(0x55799962c6e0 msgr2=0x55799962b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 shutdown_connections 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 1 -- 172.21.15.96:0/2836847186 wait complete. 2024-03-18T15:42:55.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.508+0000 7f60c5c5f740 20 client.16356 trim_cache size 0 max 16384 2024-03-18T15:42:55.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 Processor -- start 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 -- start start 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 0x557999642b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1a8e60 con 0x557999636cd0 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1aaa30 con 0x557999631b60 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1a9530 con 0x557999631170 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56486/0 (socket says 172.21.15.96:56486) 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:59984/0 (socket says 172.21.15.96:59984) 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60c161d640 1 -- 172.21.15.96:0/3712542647 learned_addr learned my addr 172.21.15.96:0/3712542647 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.528+0000 7f60bbfff640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 0x557999642b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 -- 172.21.15.96:0/3712542647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 msgr2=0x5579996323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 -- 172.21.15.96:0/3712542647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 msgr2=0x557999642b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 0x557999642b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 -- 172.21.15.96:0/3712542647 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799926a0f0 con 0x557999636cd0 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60bbfff640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 0x557999642b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60c161d640 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c630 tx=0x7f60b4005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629afdaa509cfe87 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60a67fc640 1 -- 172.21.15.96:0/3712542647 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b405e070 con 0x557999636cd0 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60a67fc640 1 -- 172.21.15.96:0/3712542647 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60b40329a0 con 0x557999636cd0 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.529+0000 7f60a67fc640 1 -- 172.21.15.96:0/3712542647 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4054040 con 0x557999636cd0 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 -- 172.21.15.96:0/3712542647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 msgr2=0x55799962b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f60b402c630 tx=0x7f60b4005b70 comp rx=0 tx=0).stop 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 -- 172.21.15.96:0/3712542647 shutdown_connections 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x55799962b070 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631b60 0x557999642b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/3712542647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631170 0x5579996323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 -- 172.21.15.96:0/3712542647 >> 172.21.15.96:0/3712542647 conn(0x55799962c6e0 msgr2=0x55799962cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 -- 172.21.15.96:0/3712542647 shutdown_connections 2024-03-18T15:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.530+0000 7f60c5c5f740 1 -- 172.21.15.96:0/3712542647 wait complete. 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 -- start start 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 Processor -- start 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x557999640490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x5579996409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1aa330 con 0x557999636cd0 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1a9c60 con 0x557999631170 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc1a6e40 con 0x557999631b60 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c5c5f740 10 client.0 _ll_register_callbacks cb 0x7f60b4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x5579996409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c161d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56492/0 (socket says 172.21.15.96:56492) 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c0e1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x557999640490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.531+0000 7f60c161d640 1 -- 172.21.15.96:0/4137804180 learned_addr learned my addr 172.21.15.96:0/4137804180 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 -- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 msgr2=0x5579996409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x5579996409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 -- 172.21.15.96:0/4137804180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 msgr2=0x557999640490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x557999640490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 -- 172.21.15.96:0/4137804180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799955d9d0 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x557999640490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f60c161d640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f60b4005e30 tx=0x7f60b402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabd4308b4067ce0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f609bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b4054070 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.532+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f60b4004460 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55799959dd90 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60b4003d40 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 10 client.16362 fetch_fsmap learned FSMap version 30 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579996ad530 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55799959dd90 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f60b405a430 con 0x557999636cd0 2024-03-18T15:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:42:55.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:42:55.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 20 client.16362 populate_metadata read hostname 'smithi096' 2024-03-18T15:42:55.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996afe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:42:55.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 4 client.16362 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:42:55.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.533+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55799951b540 con 0x557999647130 2024-03-18T15:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.534+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.534+0000 7f609bfff640 10 client.16362 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.534+0000 7f60c0e1c640 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996afe20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9569dda05d07a3f server_cookie=e235279696a32f2c in_seq=0 out_seq=0 2024-03-18T15:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f609bfff640 1 -- 172.21.15.96:0/4137804180 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55799951b540 con 0x557999647130 2024-03-18T15:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f609bfff640 10 client.16362 handle_command_reply: tid=0 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 client.16362 shutdown 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 msgr2=0x557999641990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f60b4005e30 tx=0x7f60b402c890 comp rx=0 tx=0).stop 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 shutdown_connections 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557999647130 0x5579996afe20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557999636cd0 0x557999641990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557999631b60 0x5579996409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 --2- 172.21.15.96:0/4137804180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557999631170 0x557999640490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f609bfff640 0 client.16362 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 >> 172.21.15.96:0/4137804180 conn(0x55799962c6e0 msgr2=0x55799962f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 shutdown_connections 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.535+0000 7f60c5c5f740 1 -- 172.21.15.96:0/4137804180 wait complete. 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c5c5f740 20 client.16362 trim_cache size 0 max 16384 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "a795e0da-d125-4831-b835-6249275c175d": { 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "a795e0da-d125-4831-b835-6249275c175d", 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:42:55.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:42:55.541 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:42:55.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 msgr2=0x7f60ac0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 0x7f60ac0a4d40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5579996323c0 tx=0x7f60b4032590 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 msgr2=0x7f60bc197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60b000dbc0 tx=0x7f60b000a620 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 shutdown_connections 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f60ac0a2880 0x7f60ac0a4d40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60bc101990 0x7f60bc19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60bc100fd0 0x7f60bc198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.536+0000 7f60c3080640 1 --2- 172.21.15.96:0/3893311072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60bc0ffe80 0x7f60bc197ad0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.537+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 >> 172.21.15.96:0/3893311072 conn(0x7f60bc0fb410 msgr2=0x7f60bc1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.537+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 shutdown_connections 2024-03-18T15:42:55.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:42:55.537+0000 7f60c3080640 1 -- 172.21.15.96:0/3893311072 wait complete. 2024-03-18T15:42:55.551 INFO:tasks.cephfs.filesystem:scrub status for tag:a795e0da-d125-4831-b835-6249275c175d - {'path': '/', 'tag': 'a795e0da-d125-4831-b835-6249275c175d', 'options': 'recursive,force'} 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:55.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:55 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:55 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:56.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:55 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:56 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:56 smithi193 ceph-mon[36702]: pgmap v437: 97 pgs: 97 active+clean; 306 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-03-18T15:42:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:56 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:56 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:56 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:56 smithi096 ceph-mon[31930]: pgmap v437: 97 pgs: 97 active+clean; 306 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-03-18T15:42:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:56 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:56 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:56 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:57.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:56 smithi137 ceph-mon[35868]: pgmap v437: 97 pgs: 97 active+clean; 306 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 132 op/s 2024-03-18T15:42:57.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:56 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:57.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:56 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:57.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:57 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:57.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:57 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:57.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:57 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:42:57.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:57 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:57.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:57 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:57.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:57 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:57.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:57 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:42:57.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:57 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:58.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:57 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:58.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:57 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:58.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:57 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:42:58.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:57 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:42:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:42:58 smithi193 ceph-mon[36702]: pgmap v438: 97 pgs: 97 active+clean; 305 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 931 KiB/s wr, 103 op/s 2024-03-18T15:42:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:42:58 smithi096 ceph-mon[31930]: pgmap v438: 97 pgs: 97 active+clean; 305 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 931 KiB/s wr, 103 op/s 2024-03-18T15:42:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:42:58 smithi137 ceph-mon[35868]: pgmap v438: 97 pgs: 97 active+clean; 305 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 931 KiB/s wr, 103 op/s 2024-03-18T15:43:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:00 smithi193 ceph-mon[36702]: pgmap v439: 97 pgs: 97 active+clean; 307 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-03-18T15:43:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:00 smithi096 ceph-mon[31930]: pgmap v439: 97 pgs: 97 active+clean; 307 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-03-18T15:43:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:00 smithi137 ceph-mon[35868]: pgmap v439: 97 pgs: 97 active+clean; 307 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-03-18T15:43:01.328 INFO:tasks.workunit.client.0.smithi096.stdout: 69 688510 264 513025 139 382867 552 2024-03-18T15:43:02.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:02 smithi193 ceph-mon[36702]: pgmap v440: 97 pgs: 97 active+clean; 318 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-18T15:43:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:02 smithi096 ceph-mon[31930]: pgmap v440: 97 pgs: 97 active+clean; 318 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-18T15:43:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:02 smithi137 ceph-mon[35868]: pgmap v440: 97 pgs: 97 active+clean; 318 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.5 MiB/s wr, 145 op/s 2024-03-18T15:43:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:04 smithi193 ceph-mon[36702]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-03-18T15:43:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:04 smithi096 ceph-mon[31930]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-03-18T15:43:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.966+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1357751239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141022b0 msgr2=0x7f6c14108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.966+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1357751239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141022b0 0x7f6c14108b40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009ca0 tx=0x7f6c0c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1357751239 shutdown_connections 2024-03-18T15:43:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1357751239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141022b0 0x7f6c14108b40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.979 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1357751239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c141018f0 0x7f6c14101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1357751239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c14100700 0x7f6c14100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1357751239 >> 172.21.15.96:0/1357751239 conn(0x7f6c140fbc70 msgr2=0x7f6c140fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1357751239 shutdown_connections 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1357751239 wait complete. 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 Processor -- start 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- start start 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c14100700 0x7f6c14197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 0x7f6c1419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1410c380 con 0x7f6c141018f0 2024-03-18T15:43:04.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1410c200 con 0x7f6c141022b0 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1c6ea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1410c500 con 0x7f6c14100700 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38864/0 (socket says 172.21.15.96:38864) 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1ac87640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 0x7f6c1419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c1a486640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c14100700 0x7f6c14197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.967+0000 7f6c19c85640 1 -- 172.21.15.96:0/1250761855 learned_addr learned my addr 172.21.15.96:0/1250761855 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 -- 172.21.15.96:0/1250761855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c14100700 msgr2=0x7f6c14197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c14100700 0x7f6c14197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 -- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 msgr2=0x7f6c14197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 -- 172.21.15.96:0/1250761855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1419f6a0 con 0x7f6c141022b0 2024-03-18T15:43:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1ac87640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 0x7f6c1419f030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02c630 tx=0x7f6c0c032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c0b7fe640 1 -- 172.21.15.96:0/1250761855 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c041020 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1419f930 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c0b7fe640 1 -- 172.21.15.96:0/1250761855 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0c034cd0 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1419fe40 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.968+0000 7f6c0b7fe640 1 -- 172.21.15.96:0/1250761855 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c039040 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.970+0000 7f6c0b7fe640 1 -- 172.21.15.96:0/1250761855 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f6c0c044020 con 0x7f6c141022b0 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.970+0000 7f6c1f3b1740 1 Processor -- start 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 -- start start 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 0x55602ceb8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cebb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55602ce18a40 con 0x55602ceb3340 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55602cdc1920 con 0x55602ceb3d30 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55602ce14ae0 con 0x55602ceb8ea0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c0b7fe640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 0x7f6c000a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38866/0 (socket says 172.21.15.96:38866) 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c19c85640 1 -- 172.21.15.96:0/1145972079 learned_addr learned my addr 172.21.15.96:0/1145972079 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 0x55602ceb8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1ac87640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 0x7f6c000a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cebb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c0b7fe640 1 -- 172.21.15.96:0/1250761855 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f6c0c0f0810 con 0x7f6c141022b0 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 -- 172.21.15.96:0/1145972079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 msgr2=0x55602cebb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cebb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 -- 172.21.15.96:0/1145972079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 msgr2=0x55602ceb3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c1a486640 1 -- 172.21.15.96:0/1145972079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602cdd9460 con 0x55602ceb3d30 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.971+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1a486640 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 0x55602ceb8830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009bf0 tx=0x7f6c10011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4791f07a5e694fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1ac87640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 0x7f6c000a4d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0b9be0 tx=0x7f6c0c08c070 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c08ff9640 1 -- 172.21.15.96:0/1145972079 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10009da0 con 0x55602ceb3d30 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c08ff9640 1 -- 172.21.15.96:0/1145972079 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10005020 con 0x55602ceb3d30 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c08ff9640 1 -- 172.21.15.96:0/1145972079 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10004b00 con 0x55602ceb3d30 2024-03-18T15:43:04.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1145972079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 msgr2=0x55602ceb8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 0x55602ceb8830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009bf0 tx=0x7f6c10011810 comp rx=0 tx=0).stop 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1145972079 shutdown_connections 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cebb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3d30 0x55602ceb8830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1145972079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1145972079 >> 172.21.15.96:0/1145972079 conn(0x55602ceae8b0 msgr2=0x55602ceaecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1145972079 shutdown_connections 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1145972079 wait complete. 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- start start 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 Processor -- start 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cebd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602cebdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55602cebbe50 con 0x55602ceb8ea0 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55602cdd4d70 con 0x55602ceb3340 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55602cebbfd0 con 0x55602ceb3d30 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1f3b1740 10 client.0 _ll_register_callbacks cb 0x55602cb2ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38874/0 (socket says 172.21.15.96:38874) 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c19c85640 1 -- 172.21.15.96:0/432246506 learned_addr learned my addr 172.21.15.96:0/432246506 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c1a486640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602cebdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.973+0000 7f6c19c85640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cebd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 -- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 msgr2=0x55602cebdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602cebdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 -- 172.21.15.96:0/432246506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 msgr2=0x55602cebd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cebd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 -- 172.21.15.96:0/432246506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602cb2d210 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1a486640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602cebdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c19c85640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6c04010700 tx=0x7f6c04005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572616902e8fd743 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400fa80 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f6c0400fc20 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c10005020 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c0402b050 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1f3b1740 10 client.16374 fetch_fsmap learned FSMap version 30 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55602cbf9620 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10005020 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6c040253e0 con 0x55602ceb8ea0 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1f3b1740 10 client.16374 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.974+0000 7f6c1f3b1740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1f3b1740 20 client.16374 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cec94a0 0x55602cecb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1f3b1740 4 client.16374 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55602ce25f80 con 0x55602cec94a0 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1a486640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cec94a0 0x55602cecb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6bfdffb640 10 client.16374 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.975+0000 7f6c1a486640 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cec94a0 0x55602cecb930 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d934e820de58ed server_cookie=9baebae5ea2823b8 in_seq=0 out_seq=0 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6bfdffb640 1 -- 172.21.15.96:0/432246506 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55602ce25f80 con 0x55602cec94a0 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6bfdffb640 10 client.16374 handle_command_reply: tid=0 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 client.16374 shutdown 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 msgr2=0x55602cec4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6c04010700 tx=0x7f6c04005f90 comp rx=0 tx=0).stop 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 shutdown_connections 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cec94a0 0x55602cecb930 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb8ea0 0x55602cec4bc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602cebdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6bfdffb640 0 client.16374 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/432246506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cebd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 >> 172.21.15.96:0/432246506 conn(0x55602ceae8b0 msgr2=0x55602ceadbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 shutdown_connections 2024-03-18T15:43:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/432246506 wait complete. 2024-03-18T15:43:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.976+0000 7f6c1f3b1740 20 client.16374 trim_cache size 0 max 16384 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 Processor -- start 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 -- start start 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602ceb5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb8ea0 0x55602cec4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141a8d20 con 0x55602ceb3340 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141aa8f0 con 0x55602ceb8ea0 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141a93f0 con 0x55602ceb3d30 2024-03-18T15:43:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.991+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38884/0 (socket says 172.21.15.96:38884) 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 -- 172.21.15.96:0/1884962679 learned_addr learned my addr 172.21.15.96:0/1884962679 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c1a486640 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602ceb5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb8ea0 0x55602cec4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 -- 172.21.15.96:0/1884962679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 msgr2=0x55602ceb5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602ceb5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 -- 172.21.15.96:0/1884962679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb8ea0 msgr2=0x55602cec4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb8ea0 0x55602cec4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 -- 172.21.15.96:0/1884962679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602cea0140 con 0x55602ceb3340 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6c19c85640 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c040106d0 tx=0x7f6c04001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cb93f691909f21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6bfffff640 1 -- 172.21.15.96:0/1884962679 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040255e0 con 0x55602ceb3340 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6bfffff640 1 -- 172.21.15.96:0/1884962679 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c04037070 con 0x55602ceb3340 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.992+0000 7f6bfffff640 1 -- 172.21.15.96:0/1884962679 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0400fae0 con 0x55602ceb3340 2024-03-18T15:43:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1884962679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 msgr2=0x55602ceb4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6c040106d0 tx=0x7f6c04001e70 comp rx=0 tx=0).stop 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1884962679 shutdown_connections 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb8ea0 0x55602cec4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb3d30 0x55602ceb5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/1884962679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3340 0x55602ceb4ff0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1884962679 >> 172.21.15.96:0/1884962679 conn(0x55602ceae8b0 msgr2=0x55602cea9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1884962679 shutdown_connections 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/1884962679 wait complete. 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- start start 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 Processor -- start 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cec20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cec35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141aa1f0 con 0x55602ceb3d30 2024-03-18T15:43:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141a9b20 con 0x55602ceb3340 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141a6d00 con 0x55602ceb8ea0 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.994+0000 7f6c1f3b1740 10 client.0 _ll_register_callbacks cb 0x7f6c04013060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cec35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38896/0 (socket says 172.21.15.96:38896) 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c19c85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cec35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:54310/0 (socket says 172.21.15.96:54310) 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 -- 172.21.15.96:0/123111295 learned_addr learned my addr 172.21.15.96:0/123111295 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c19c85640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cec20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 -- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 msgr2=0x55602cec35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cec35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 -- 172.21.15.96:0/123111295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 msgr2=0x55602cec20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cec20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 -- 172.21.15.96:0/123111295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602cec3bc0 con 0x55602ceb3d30 2024-03-18T15:43:05.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1a486640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6c100057c0 tx=0x7f6c10005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd7342aabe87a3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1002ba30 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f6c1002bbd0 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c04037070 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c1003b050 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 10 client.16380 fetch_fsmap learned FSMap version 30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55602cec4f40 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c04037070 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6c10019780 con 0x55602ceb3d30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 10 client.16380 fetch_fsmap finished waiting for FSMap version 30 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 20 client.16380 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cf2fef0 0x55602cf32310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:05.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 4 client.16380 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:43:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55602ce202d0 con 0x55602cf2fef0 2024-03-18T15:43:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c19c85640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cf2fef0 0x55602cf32310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6bfdffb640 10 client.16380 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:04.995+0000 7f6c19c85640 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cf2fef0 0x55602cf32310 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9887ec3da8c094 server_cookie=7f3e4423a4dfe92c in_seq=0 out_seq=0 2024-03-18T15:43:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6bfdffb640 1 -- 172.21.15.96:0/123111295 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55602ce202d0 con 0x55602cf2fef0 2024-03-18T15:43:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6bfdffb640 10 client.16380 handle_command_reply: tid=0 2024-03-18T15:43:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 client.16380 shutdown 2024-03-18T15:43:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 msgr2=0x55602cec2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6c100057c0 tx=0x7f6c10005750 comp rx=0 tx=0).stop 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 shutdown_connections 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55602ceb8ea0 0x55602cec35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55602cf2fef0 0x55602cf32310 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55602ceb3d30 0x55602cec2610 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6bfdffb640 0 client.16380 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 --2- 172.21.15.96:0/123111295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55602ceb3340 0x55602cec20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 >> 172.21.15.96:0/123111295 conn(0x55602ceae8b0 msgr2=0x55602cebd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 shutdown_connections 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 1 -- 172.21.15.96:0/123111295 wait complete. 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1f3b1740 20 client.16380 trim_cache size 0 max 16384 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:43:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 msgr2=0x7f6c000a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 0x7f6c000a4d60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0b9be0 tx=0x7f6c0c08c070 comp rx=0 tx=0).stop 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 msgr2=0x7f6c1419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.004+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 0x7f6c1419f030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c02c630 tx=0x7f6c0c032f20 comp rx=0 tx=0).stop 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 shutdown_connections 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f6c000a28a0 0x7f6c000a4d60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c141022b0 0x7f6c1419f030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c141018f0 0x7f6c14197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 --2- 172.21.15.96:0/1250761855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c14100700 0x7f6c14197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 >> 172.21.15.96:0/1250761855 conn(0x7f6c140fbc70 msgr2=0x7f6c140fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 shutdown_connections 2024-03-18T15:43:05.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:05.005+0000 7f6c1c6ea640 1 -- 172.21.15.96:0/1250761855 wait complete. 2024-03-18T15:43:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:04 smithi137 ceph-mon[35868]: pgmap v441: 97 pgs: 97 active+clean; 345 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 3.9 MiB/s wr, 288 op/s 2024-03-18T15:43:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:06 smithi193 ceph-mon[36702]: pgmap v442: 97 pgs: 97 active+clean; 357 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 352 op/s 2024-03-18T15:43:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:06 smithi096 ceph-mon[31930]: pgmap v442: 97 pgs: 97 active+clean; 357 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 352 op/s 2024-03-18T15:43:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:06 smithi137 ceph-mon[35868]: pgmap v442: 97 pgs: 97 active+clean; 357 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 352 op/s 2024-03-18T15:43:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:08 smithi193 ceph-mon[36702]: pgmap v443: 97 pgs: 97 active+clean; 357 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.7 MiB/s wr, 339 op/s 2024-03-18T15:43:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:08 smithi096 ceph-mon[31930]: pgmap v443: 97 pgs: 97 active+clean; 357 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.7 MiB/s wr, 339 op/s 2024-03-18T15:43:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:08 smithi137 ceph-mon[35868]: pgmap v443: 97 pgs: 97 active+clean; 357 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 4.7 MiB/s wr, 339 op/s 2024-03-18T15:43:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:10 smithi193 ceph-mon[36702]: pgmap v444: 97 pgs: 97 active+clean; 358 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-03-18T15:43:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:10 smithi096 ceph-mon[31930]: pgmap v444: 97 pgs: 97 active+clean; 358 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-03-18T15:43:11.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:10 smithi137 ceph-mon[35868]: pgmap v444: 97 pgs: 97 active+clean; 358 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 351 op/s 2024-03-18T15:43:11.328 INFO:tasks.workunit.client.0.smithi096.stdout: 71 372704 181 271908 190 199100 527 2024-03-18T15:43:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:12 smithi193 ceph-mon[36702]: pgmap v445: 97 pgs: 97 active+clean; 358 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.7 MiB/s wr, 336 op/s 2024-03-18T15:43:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:12 smithi096 ceph-mon[31930]: pgmap v445: 97 pgs: 97 active+clean; 358 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.7 MiB/s wr, 336 op/s 2024-03-18T15:43:13.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:12 smithi137 ceph-mon[35868]: pgmap v445: 97 pgs: 97 active+clean; 358 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 4.7 MiB/s wr, 336 op/s 2024-03-18T15:43:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:14 smithi193 ceph-mon[36702]: pgmap v446: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-03-18T15:43:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:14 smithi096 ceph-mon[31930]: pgmap v446: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-03-18T15:43:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:14 smithi137 ceph-mon[35868]: pgmap v446: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 294 op/s 2024-03-18T15:43:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:15 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:15 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:16.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:15 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:16 smithi193 ceph-mon[36702]: pgmap v447: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-18T15:43:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:16 smithi096 ceph-mon[31930]: pgmap v447: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-18T15:43:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:16 smithi137 ceph-mon[35868]: pgmap v447: 97 pgs: 97 active+clean; 361 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 166 op/s 2024-03-18T15:43:18.116 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 -- 172.21.15.96:0/243231777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101320 msgr2=0x7f12d8108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 --2- 172.21.15.96:0/243231777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101320 0x7f12d8108bc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009ca0 tx=0x7f12d002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 -- 172.21.15.96:0/243231777 shutdown_connections 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 --2- 172.21.15.96:0/243231777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101320 0x7f12d8108bc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 --2- 172.21.15.96:0/243231777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8100960 0x7f12d8100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 --2- 172.21.15.96:0/243231777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8101d10 0x7f12d8102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 -- 172.21.15.96:0/243231777 >> 172.21.15.96:0/243231777 conn(0x7f12d80fb430 msgr2=0x7f12d80fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 -- 172.21.15.96:0/243231777 shutdown_connections 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.113+0000 7f12dd2de640 1 -- 172.21.15.96:0/243231777 wait complete. 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 Processor -- start 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 -- start start 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8101320 0x7f12d81096f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 0x7f12d8109c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d500 con 0x7f12d8101d10 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d380 con 0x7f12d8100960 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12dd2de640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12d810d680 con 0x7f12d8101320 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 0x7f12d8109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8101320 0x7f12d81096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:36690/0 (socket says 172.21.15.96:36690) 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 0x7f12d8109c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39918/0 (socket says 172.21.15.96:39918) 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.114+0000 7f12d6ffd640 1 -- 172.21.15.96:0/1443328602 learned_addr learned my addr 172.21.15.96:0/1443328602 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 -- 172.21.15.96:0/1443328602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8101320 msgr2=0x7f12d81096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8101320 0x7f12d81096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 -- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 msgr2=0x7f12d8109c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 0x7f12d8109c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 -- 172.21.15.96:0/1443328602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d810a4c0 con 0x7f12d8100960 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9c0 tx=0x7f12c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12bffff640 1 -- 172.21.15.96:0/1443328602 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c800cd80 con 0x7f12d8100960 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12bffff640 1 -- 172.21.15.96:0/1443328602 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12c80035f0 con 0x7f12d8100960 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d81a1bb0 con 0x7f12d8100960 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12bffff640 1 -- 172.21.15.96:0/1443328602 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8010690 con 0x7f12d8100960 2024-03-18T15:43:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.115+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d81a2100 con 0x7f12d8100960 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12bffff640 1 -- 172.21.15.96:0/1443328602 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f12c8010830 con 0x7f12d8100960 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 Processor -- start 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 -- start start 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88ae4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88aec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da88a49a40 con 0x55da88ae4d30 2024-03-18T15:43:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da889f2920 con 0x55da88ae4340 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12dff9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da88a45ae0 con 0x55da88ae9ea0 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39932/0 (socket says 172.21.15.96:39932) 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d6ffd640 1 -- 172.21.15.96:0/2669254913 learned_addr learned my addr 172.21.15.96:0/2669254913 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d67fc640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88aec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d77fe640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88ae4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12bffff640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 0x7f12c00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.118+0000 7f12d77fe640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 0x7f12c00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 -- 172.21.15.96:0/2669254913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 msgr2=0x55da88aec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88aec330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 -- 172.21.15.96:0/2669254913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 msgr2=0x55da88ae4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88ae4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 -- 172.21.15.96:0/2669254913 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da88a0a460 con 0x55da88ae4d30 2024-03-18T15:43:18.125 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d67fc640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88aec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12bffff640 1 -- 172.21.15.96:0/1443328602 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f12c80ceed0 con 0x7f12d8100960 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f12c8097d10 tx=0x7f12c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf39249ec46e703 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12bd7fa640 1 -- 172.21.15.96:0/2669254913 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80cce60 con 0x55da88ae4d30 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12d77fe640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 0x7f12c00a4d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12bd7fa640 1 -- 172.21.15.96:0/2669254913 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12c80a2aa0 con 0x55da88ae4d30 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.119+0000 7f12bd7fa640 1 -- 172.21.15.96:0/2669254913 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c809f990 con 0x55da88ae4d30 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- 172.21.15.96:0/2669254913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 msgr2=0x55da88ae9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f12c8097d10 tx=0x7f12c8017000 comp rx=0 tx=0).stop 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- 172.21.15.96:0/2669254913 shutdown_connections 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88aec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88ae9830 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2669254913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88ae4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- 172.21.15.96:0/2669254913 >> 172.21.15.96:0/2669254913 conn(0x55da88adf8b0 msgr2=0x55da88adfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- 172.21.15.96:0/2669254913 shutdown_connections 2024-03-18T15:43:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- 172.21.15.96:0/2669254913 wait complete. 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 -- start start 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.120+0000 7f12dff9c740 1 Processor -- start 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4d30 0x55da88aeeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da88aecdf0 con 0x55da88ae4340 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55da88a05d70 con 0x55da88ae4d30 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da88aecf70 con 0x55da88ae9ea0 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12dff9c740 10 client.0 _ll_register_callbacks cb 0x55da8875da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39944/0 (socket says 172.21.15.96:39944) 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 -- 172.21.15.96:0/3479348133 learned_addr learned my addr 172.21.15.96:0/3479348133 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d67fc640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4d30 0x55da88aeeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d77fe640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 -- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 msgr2=0x55da88af5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 -- 172.21.15.96:0/3479348133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4d30 msgr2=0x55da88aeeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4d30 0x55da88aeeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 -- 172.21.15.96:0/3479348133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8875e210 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f12c80cc9f0 tx=0x7f12c80cf240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71acd105caec3a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.121+0000 7f12baff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80cce60 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f12c80d20b0 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da88a56f80 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12c809b3f0 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 10 client.16392 fetch_fsmap learned FSMap version 31 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da8882a620 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55da88a56f80 con 0x55da88ae4340 2024-03-18T15:43:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f12c80860d0 con 0x55da88ae4340 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 10 client.16392 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 20 client.16392 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88afa050 0x55da88afc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 4 client.16392 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.122+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da88a512d0 con 0x55da88afa050 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.123+0000 7f12d67fc640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88afa050 0x55da88afc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.123+0000 7f12d67fc640 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88afa050 0x55da88afc460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27270e8e4dbd8a81 server_cookie=793c6c957709a851 in_seq=0 out_seq=0 2024-03-18T15:43:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.123+0000 7f12baff5640 10 client.16392 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12baff5640 1 -- 172.21.15.96:0/3479348133 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55da88a512d0 con 0x55da88afa050 2024-03-18T15:43:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12baff5640 10 client.16392 handle_command_reply: tid=0 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 client.16392 shutdown 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 msgr2=0x55da88aee5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f12c80cc9f0 tx=0x7f12c80cf240 comp rx=0 tx=0).stop 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 shutdown_connections 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4d30 0x55da88aeeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88afa050 0x55da88afc460 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 --2- 172.21.15.96:0/3479348133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4340 0x55da88aee5e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12baff5640 0 client.16392 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 >> 172.21.15.96:0/3479348133 conn(0x55da88adf8b0 msgr2=0x55da88adebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 shutdown_connections 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 1 -- 172.21.15.96:0/3479348133 wait complete. 2024-03-18T15:43:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.125+0000 7f12dff9c740 20 client.16392 trim_cache size 0 max 16384 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 Processor -- start 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 -- start start 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88aee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae4d30 0x55da88af5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81ab060 con 0x55da88ae9ea0 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81acc60 con 0x55da88ae4340 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12dff9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81ab730 con 0x55da88ae4d30 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88aee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae4d30 0x55da88af5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39954/0 (socket says 172.21.15.96:39954) 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88aee900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:36724/0 (socket says 172.21.15.96:36724) 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.156+0000 7f12d77fe640 1 -- 172.21.15.96:0/4240135964 learned_addr learned my addr 172.21.15.96:0/4240135964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 -- 172.21.15.96:0/4240135964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae4d30 msgr2=0x55da88af5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae4d30 0x55da88af5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 -- 172.21.15.96:0/4240135964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 msgr2=0x55da88aee900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88aee900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 -- 172.21.15.96:0/4240135964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da88afcae0 con 0x55da88ae9ea0 2024-03-18T15:43:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12d77fe640 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f12d0037040 tx=0x7f12d0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0f680a72b3cfcc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12bcff9640 1 -- 172.21.15.96:0/4240135964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0039040 con 0x55da88ae9ea0 2024-03-18T15:43:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12bcff9640 1 -- 172.21.15.96:0/4240135964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d0054070 con 0x55da88ae9ea0 2024-03-18T15:43:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.157+0000 7f12bcff9640 1 -- 172.21.15.96:0/4240135964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d004f6b0 con 0x55da88ae9ea0 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- 172.21.15.96:0/4240135964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 msgr2=0x55da88ade320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f12d0037040 tx=0x7f12d0009360 comp rx=0 tx=0).stop 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- 172.21.15.96:0/4240135964 shutdown_connections 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae9ea0 0x55da88ade320 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae4d30 0x55da88af5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 --2- 172.21.15.96:0/4240135964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88aee900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- 172.21.15.96:0/4240135964 >> 172.21.15.96:0/4240135964 conn(0x55da88adf8b0 msgr2=0x55da88aee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- 172.21.15.96:0/4240135964 shutdown_connections 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- 172.21.15.96:0/4240135964 wait complete. 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 -- start start 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.158+0000 7f12dff9c740 1 Processor -- start 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88af3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81ac560 con 0x55da88ae4d30 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81abe90 con 0x55da88ae4340 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12d81a9040 con 0x55da88ae9ea0 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 10 client.0 _ll_register_callbacks cb 0x7f12c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39964/0 (socket says 172.21.15.96:39964) 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 -- 172.21.15.96:0/2863720309 learned_addr learned my addr 172.21.15.96:0/2863720309 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88af3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d77fe640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 -- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 msgr2=0x55da88af4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 -- 172.21.15.96:0/2863720309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 msgr2=0x55da88af3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88af3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 -- 172.21.15.96:0/2863720309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da88ad1140 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88af3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12d67fc640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f12cc004700 tx=0x7f12cc020630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13ab0caa3716c9b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12baff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc024030 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f12cc005020 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12d0054070 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.159+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f12cc020820 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 10 client.16398 fetch_fsmap learned FSMap version 31 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55da88b60830 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d0054070 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7f12cc0047a0 con 0x55da88ae4d30 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 10 client.16398 fetch_fsmap finished waiting for FSMap version 31 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:18.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 20 client.16398 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88b61290 0x55da88b636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 4 client.16398 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da889cf770 con 0x55da88b61290 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.160+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88b61290 0x55da88b636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.161+0000 7f12baff5640 10 client.16398 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:18.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.161+0000 7f12d6ffd640 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88b61290 0x55da88b636b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8c6242e0e0241c server_cookie=d971765dea61c0b4 in_seq=0 out_seq=0 2024-03-18T15:43:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12baff5640 1 -- 172.21.15.96:0/2863720309 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da889cf770 con 0x55da88b61290 2024-03-18T15:43:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12baff5640 10 client.16398 handle_command_reply: tid=0 2024-03-18T15:43:18.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 client.16398 shutdown 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 msgr2=0x55da88af3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f12cc004700 tx=0x7f12cc020630 comp rx=0 tx=0).stop 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 shutdown_connections 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da88ae9ea0 0x55da88af4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55da88b61290 0x55da88b636b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da88ae4d30 0x55da88af3cc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 --2- 172.21.15.96:0/2863720309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55da88ae4340 0x55da88af3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12baff5640 0 client.16398 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 >> 172.21.15.96:0/2863720309 conn(0x55da88adf8b0 msgr2=0x55da88ae2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.487+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 shutdown_connections 2024-03-18T15:43:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dff9c740 1 -- 172.21.15.96:0/2863720309 wait complete. 2024-03-18T15:43:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dff9c740 20 client.16398 trim_cache size 0 max 16384 2024-03-18T15:43:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:43:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:43:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:43:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:43:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 msgr2=0x7f12c00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 0x7f12c00a4d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 msgr2=0x7f12d810b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9c0 tx=0x7f12c800ee80 comp rx=0 tx=0).stop 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 shutdown_connections 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f12c00a28a0 0x7f12c00a4d60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12d8101d10 0x7f12d8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d8101320 0x7f12d81096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 --2- 172.21.15.96:0/1443328602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f12d8100960 0x7f12d810b0b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 >> 172.21.15.96:0/1443328602 conn(0x7f12d80fb430 msgr2=0x7f12d8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 shutdown_connections 2024-03-18T15:43:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:18.488+0000 7f12dd2de640 1 -- 172.21.15.96:0/1443328602 wait complete. 2024-03-18T15:43:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:18 smithi193 ceph-mon[36702]: pgmap v448: 97 pgs: 97 active+clean; 360 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 705 KiB/s wr, 90 op/s 2024-03-18T15:43:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:18 smithi096 ceph-mon[31930]: pgmap v448: 97 pgs: 97 active+clean; 360 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 705 KiB/s wr, 90 op/s 2024-03-18T15:43:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:18 smithi137 ceph-mon[35868]: pgmap v448: 97 pgs: 97 active+clean; 360 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 705 KiB/s wr, 90 op/s 2024-03-18T15:43:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:20 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:20 smithi193 ceph-mon[36702]: pgmap v449: 97 pgs: 97 active+clean; 364 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T15:43:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:20 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:20 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:20 smithi096 ceph-mon[31930]: pgmap v449: 97 pgs: 97 active+clean; 364 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T15:43:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:20 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:20 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:20 smithi137 ceph-mon[35868]: pgmap v449: 97 pgs: 97 active+clean; 364 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-03-18T15:43:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:20 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:43:21.328 INFO:tasks.workunit.client.0.smithi096.stdout: 79 1393703 485 1034298 338 771454 1101 2024-03-18T15:43:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:22 smithi193 ceph-mon[36702]: pgmap v450: 97 pgs: 97 active+clean; 364 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-03-18T15:43:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:22 smithi096 ceph-mon[31930]: pgmap v450: 97 pgs: 97 active+clean; 364 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-03-18T15:43:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:22 smithi137 ceph-mon[35868]: pgmap v450: 97 pgs: 97 active+clean; 364 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 1.0 MiB/s wr, 117 op/s 2024-03-18T15:43:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:24 smithi096 ceph-mon[31930]: pgmap v451: 97 pgs: 97 active+clean; 365 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T15:43:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:24 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:43:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:24 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:43:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:24 smithi137 ceph-mon[35868]: pgmap v451: 97 pgs: 97 active+clean; 365 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T15:43:25.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:24 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:43:25.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:24 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:43:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:24 smithi193 ceph-mon[36702]: pgmap v451: 97 pgs: 97 active+clean; 365 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-03-18T15:43:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:24 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:43:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:24 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:43:25.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:43:25.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:43:25.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.759+0000 7fcc9b987640 1 -- 172.21.15.96:0/863177481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 msgr2=0x7fcc94104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.759+0000 7fcc9b987640 1 --2- 172.21.15.96:0/863177481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc94104e70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009900 tx=0x7fcc8402bc40 comp rx=0 tx=0).stop 2024-03-18T15:43:25.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 -- 172.21.15.96:0/863177481 shutdown_connections 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 --2- 172.21.15.96:0/863177481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc941053b0 0x7fcc94105830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 --2- 172.21.15.96:0/863177481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc94104e70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 --2- 172.21.15.96:0/863177481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc94100090 0x7fcc941024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 -- 172.21.15.96:0/863177481 >> 172.21.15.96:0/863177481 conn(0x7fcc940fbc90 msgr2=0x7fcc940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.760+0000 7fcc9b987640 1 -- 172.21.15.96:0/863177481 shutdown_connections 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.761+0000 7fcc9b987640 1 -- 172.21.15.96:0/863177481 wait complete. 2024-03-18T15:43:25.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.762+0000 7fcc9b987640 1 Processor -- start 2024-03-18T15:43:25.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.762+0000 7fcc9b987640 1 -- start start 2024-03-18T15:43:25.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.762+0000 7fcc9b987640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 0x7fcc9419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc9b987640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc9419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc9b987640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 0x7fcc941a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc9b987640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94113b30 con 0x7fcc94102a00 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc99723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 0x7fcc9419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc9b987640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941139b0 con 0x7fcc94100090 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc99723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 0x7fcc9419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41854/0 (socket says 172.21.15.96:41854) 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc99723640 1 -- 172.21.15.96:0/281946418 learned_addr learned my addr 172.21.15.96:0/281946418 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 0x7fcc941a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.763+0000 7fcc98f22640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc9419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.764+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94113cb0 con 0x7fcc941053b0 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.765+0000 7fcc99f24640 1 -- 172.21.15.96:0/281946418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 msgr2=0x7fcc9419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.765+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 0x7fcc9419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.765+0000 7fcc99f24640 1 -- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 msgr2=0x7fcc9419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.765+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc9419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.765+0000 7fcc99f24640 1 -- 172.21.15.96:0/281946418 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc941a5dc0 con 0x7fcc941053b0 2024-03-18T15:43:25.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.766+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 0x7fcc941a5750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00cab0 tx=0x7fcc8c00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.768+0000 7fcc8a7fc640 1 -- 172.21.15.96:0/281946418 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00ae20 con 0x7fcc941053b0 2024-03-18T15:43:25.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.768+0000 7fcc8a7fc640 1 -- 172.21.15.96:0/281946418 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8c00f930 con 0x7fcc941053b0 2024-03-18T15:43:25.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.768+0000 7fcc8a7fc640 1 -- 172.21.15.96:0/281946418 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c018ad0 con 0x7fcc941053b0 2024-03-18T15:43:25.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.768+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc941a60b0 con 0x7fcc941053b0 2024-03-18T15:43:25.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.769+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941a6580 con 0x7fcc941053b0 2024-03-18T15:43:25.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.772+0000 7fcc9e574740 1 Processor -- start 2024-03-18T15:43:25.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.774+0000 7fcc9e574740 1 -- start start 2024-03-18T15:43:25.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.774+0000 7fcc8a7fc640 1 -- 172.21.15.96:0/281946418 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcc8c018cf0 con 0x7fcc941053b0 2024-03-18T15:43:25.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba498590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4a0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557ba3fd500 con 0x5557ba49dcd0 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5557ba3a5be0 con 0x5557ba498b60 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc9e574740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557ba3f95a0 con 0x5557ba498170 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc99723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.775+0000 7fcc8a7fc640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 0x7fcc800a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41866/0 (socket says 172.21.15.96:41866) 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 -- 172.21.15.96:0/3036332135 learned_addr learned my addr 172.21.15.96:0/3036332135 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc8a7fc640 1 -- 172.21.15.96:0/281946418 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcc8c0cedc0 con 0x7fcc941053b0 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc98f22640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba498590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc98f22640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 0x7fcc800a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 -- 172.21.15.96:0/3036332135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 msgr2=0x5557ba498590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba498590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 -- 172.21.15.96:0/3036332135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 msgr2=0x5557ba4a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc99723640 1 -- 172.21.15.96:0/3036332135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ba3b94b0 con 0x5557ba498b60 2024-03-18T15:43:25.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.776+0000 7fcc98f22640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:43:25.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.777+0000 7fcc99723640 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009b20 tx=0x7fcc9000e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0799117d8664d36 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.777+0000 7fcc99f24640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 0x7fcc800a4ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c097b30 tx=0x7fcc8c018570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.777+0000 7fcc7f7fe640 1 -- 172.21.15.96:0/3036332135 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90017070 con 0x5557ba498b60 2024-03-18T15:43:25.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.777+0000 7fcc7f7fe640 1 -- 172.21.15.96:0/3036332135 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9000ec10 con 0x5557ba498b60 2024-03-18T15:43:25.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.777+0000 7fcc7f7fe640 1 -- 172.21.15.96:0/3036332135 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90012630 con 0x5557ba498b60 2024-03-18T15:43:25.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.778+0000 7fcc9e574740 1 -- 172.21.15.96:0/3036332135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 msgr2=0x5557ba49d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.778+0000 7fcc9e574740 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009b20 tx=0x7fcc9000e1d0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.779+0000 7fcc9e574740 1 -- 172.21.15.96:0/3036332135 shutdown_connections 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.779+0000 7fcc9e574740 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.779+0000 7fcc9e574740 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498b60 0x5557ba49d660 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.779+0000 7fcc9e574740 1 --2- 172.21.15.96:0/3036332135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba498590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.780+0000 7fcc9e574740 1 -- 172.21.15.96:0/3036332135 >> 172.21.15.96:0/3036332135 conn(0x5557ba4936e0 msgr2=0x5557ba493ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.781+0000 7fcc9e574740 1 -- 172.21.15.96:0/3036332135 shutdown_connections 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.781+0000 7fcc9e574740 1 -- 172.21.15.96:0/3036332135 wait complete. 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.781+0000 7fcc9e574740 1 -- start start 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.782+0000 7fcc9e574740 1 Processor -- start 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.782+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.782+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc9e574740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5557ba4a0c80 con 0x5557ba498b60 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc9e574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5557ba3b9160 con 0x5557ba49dcd0 2024-03-18T15:43:25.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc9e574740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557ba4a0e00 con 0x5557ba498170 2024-03-18T15:43:25.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc9e574740 10 client.0 _ll_register_callbacks cb 0x5557ba39d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:25.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc99723640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc99723640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39902/0 (socket says 172.21.15.96:39902) 2024-03-18T15:43:25.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.783+0000 7fcc99723640 1 -- 172.21.15.96:0/1768943044 learned_addr learned my addr 172.21.15.96:0/1768943044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:25.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.784+0000 7fcc98f22640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.784+0000 7fcc98f22640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 -- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 msgr2=0x5557ba4a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 -- 172.21.15.96:0/1768943044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 msgr2=0x5557ba4a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 -- 172.21.15.96:0/1768943044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ba1db7d0 con 0x5557ba49dcd0 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.786+0000 7fcc98f22640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009850 tx=0x7fcc84009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9405e1cdcc642f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.787+0000 7fcc7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:25.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.788+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84004ad0 con 0x5557ba49dcd0 2024-03-18T15:43:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.788+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcc84005000 con 0x5557ba49dcd0 2024-03-18T15:43:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.788+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc9000ec10 con 0x5557ba49dcd0 2024-03-18T15:43:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.789+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc84004c70 con 0x5557ba49dcd0 2024-03-18T15:43:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.790+0000 7fcc9e574740 10 client.26198 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:25.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.790+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5557ba1113f0 con 0x5557ba49dcd0 2024-03-18T15:43:25.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.792+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc9000ec10 con 0x5557ba49dcd0 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.792+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc8404b060 con 0x5557ba49dcd0 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 10 client.26198 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 20 client.26198 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba4ae120 0x5557ba4b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 4 client.26198 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:25.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ba40aa40 con 0x5557ba4ae120 2024-03-18T15:43:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.793+0000 7fcc99723640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba4ae120 0x5557ba4b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.794+0000 7fcc99723640 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba4ae120 0x5557ba4b0580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935b89ac6feb740f server_cookie=f1c9fc8740a01fd2 in_seq=0 out_seq=0 2024-03-18T15:43:25.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.794+0000 7fcc7cff9640 10 client.26198 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:25.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.797+0000 7fcc7cff9640 1 -- 172.21.15.96:0/1768943044 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5557ba40aa40 con 0x5557ba4ae120 2024-03-18T15:43:25.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.797+0000 7fcc7cff9640 10 client.26198 handle_command_reply: tid=0 2024-03-18T15:43:25.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.797+0000 7fcc9e574740 1 client.26198 shutdown 2024-03-18T15:43:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.801+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 msgr2=0x5557ba4a9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.801+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009850 tx=0x7fcc84009260 comp rx=0 tx=0).stop 2024-03-18T15:43:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 shutdown_connections 2024-03-18T15:43:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9c60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba4ae120 0x5557ba4b0580 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 --2- 172.21.15.96:0/1768943044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 >> 172.21.15.96:0/1768943044 conn(0x5557ba4936e0 msgr2=0x5557ba492900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.802+0000 7fcc7cff9640 0 client.26198 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.803+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 shutdown_connections 2024-03-18T15:43:25.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.806+0000 7fcc9e574740 1 -- 172.21.15.96:0/1768943044 wait complete. 2024-03-18T15:43:25.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.807+0000 7fcc9e574740 20 client.26198 trim_cache size 0 max 16384 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 Processor -- start 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 -- start start 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498b60 0x5557ba4a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941af420 con 0x5557ba49dcd0 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b0ff0 con 0x5557ba498170 2024-03-18T15:43:25.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.846+0000 7fcc9e574740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941afaf0 con 0x5557ba498b60 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41882/0 (socket says 172.21.15.96:41882) 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 -- 172.21.15.96:0/4213325655 learned_addr learned my addr 172.21.15.96:0/4213325655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc99723640 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498b60 0x5557ba4a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 -- 172.21.15.96:0/4213325655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498b60 msgr2=0x5557ba4a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498b60 0x5557ba4a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 -- 172.21.15.96:0/4213325655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 msgr2=0x5557ba4b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.847+0000 7fcc98f22640 1 -- 172.21.15.96:0/4213325655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ba0d10f0 con 0x5557ba498170 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.848+0000 7fcc98f22640 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcc84001eb0 tx=0x7fcc8402e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae43fed92b1375ab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.848+0000 7fcc7effd640 1 -- 172.21.15.96:0/4213325655 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84004ad0 con 0x5557ba498170 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.848+0000 7fcc7effd640 1 -- 172.21.15.96:0/4213325655 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc84003ad0 con 0x5557ba498170 2024-03-18T15:43:25.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.849+0000 7fcc7effd640 1 -- 172.21.15.96:0/4213325655 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84034440 con 0x5557ba498170 2024-03-18T15:43:25.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.850+0000 7fcc9e574740 1 -- 172.21.15.96:0/4213325655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 msgr2=0x5557ba497050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.850+0000 7fcc9e574740 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcc84001eb0 tx=0x7fcc8402e040 comp rx=0 tx=0).stop 2024-03-18T15:43:25.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.851+0000 7fcc9e574740 1 -- 172.21.15.96:0/4213325655 shutdown_connections 2024-03-18T15:43:25.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.851+0000 7fcc9e574740 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba49dcd0 0x5557ba4b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.851+0000 7fcc9e574740 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498b60 0x5557ba4a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.851+0000 7fcc9e574740 1 --2- 172.21.15.96:0/4213325655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba498170 0x5557ba497050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.851+0000 7fcc9e574740 1 -- 172.21.15.96:0/4213325655 >> 172.21.15.96:0/4213325655 conn(0x5557ba4936e0 msgr2=0x5557ba496940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.852+0000 7fcc9e574740 1 -- 172.21.15.96:0/4213325655 shutdown_connections 2024-03-18T15:43:25.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.852+0000 7fcc9e574740 1 -- 172.21.15.96:0/4213325655 wait complete. 2024-03-18T15:43:25.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.852+0000 7fcc9e574740 1 -- start start 2024-03-18T15:43:25.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.852+0000 7fcc9e574740 1 Processor -- start 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.853+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.853+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.853+0000 7fcc9e574740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.853+0000 7fcc98f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.853+0000 7fcc98f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48042/0 (socket says 172.21.15.96:48042) 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc98f22640 1 -- 172.21.15.96:0/16949209 learned_addr learned my addr 172.21.15.96:0/16949209 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b0920 con 0x5557ba498b60 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b0250 con 0x5557ba49dcd0 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941ad400 con 0x5557ba498170 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc98f22640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc9e574740 10 client.0 _ll_register_callbacks cb 0x7fcc84009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 -- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 msgr2=0x5557ba4a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 -- 172.21.15.96:0/16949209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 msgr2=0x5557ba4a9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc99723640 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ba3c49d0 con 0x5557ba498b60 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.854+0000 7fcc98f22640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc84003ad0 con 0x5557ba498b60 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc99723640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a7ff0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcc90004700 tx=0x7fcc90009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9565cfbe48beed5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc7cff9640 10 client.16410 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc7cff9640 1 -- 172.21.15.96:0/16949209 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90034050 con 0x5557ba498b60 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc7cff9640 1 -- 172.21.15.96:0/16949209 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcc90026d20 con 0x5557ba498b60 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc7cff9640 1 -- 172.21.15.96:0/16949209 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc90017040 con 0x5557ba498b60 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc9e574740 10 client.16410 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:25.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.855+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5557ba514930 con 0x5557ba498b60 2024-03-18T15:43:25.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc7cff9640 1 -- 172.21.15.96:0/16949209 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fcc900263a0 con 0x5557ba498b60 2024-03-18T15:43:25.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 10 client.16410 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 20 client.16410 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba514f40 0x5557ba517360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 4 client.16410 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.856+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557ba404d90 con 0x5557ba514f40 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.857+0000 7fcc98f22640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba514f40 0x5557ba517360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.858+0000 7fcc7cff9640 10 client.16410 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:25.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.858+0000 7fcc98f22640 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba514f40 0x5557ba517360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e855633eda9e4c4 server_cookie=c963423201941784 in_seq=0 out_seq=0 2024-03-18T15:43:25.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc7cff9640 1 -- 172.21.15.96:0/16949209 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5557ba404d90 con 0x5557ba514f40 2024-03-18T15:43:25.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc7cff9640 10 client.16410 handle_command_reply: tid=0 2024-03-18T15:43:25.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 client.16410 shutdown 2024-03-18T15:43:25.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 msgr2=0x5557ba4a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a7ff0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcc90004700 tx=0x7fcc90009760 comp rx=0 tx=0).stop 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 shutdown_connections 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5557ba49dcd0 0x5557ba4a9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5557ba514f40 0x5557ba517360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5557ba498b60 0x5557ba4a7ff0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 --2- 172.21.15.96:0/16949209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5557ba498170 0x5557ba4a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 >> 172.21.15.96:0/16949209 conn(0x5557ba4936e0 msgr2=0x5557ba4971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.864+0000 7fcc7cff9640 0 client.16410 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.865+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 shutdown_connections 2024-03-18T15:43:25.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.865+0000 7fcc9e574740 1 -- 172.21.15.96:0/16949209 wait complete. 2024-03-18T15:43:25.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.865+0000 7fcc9e574740 20 client.16410 trim_cache size 0 max 16384 2024-03-18T15:43:25.867 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:43:25.867 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:43:25.867 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:43:25.867 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:43:25.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 msgr2=0x7fcc800a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 0x7fcc800a4ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c097b30 tx=0x7fcc8c018570 comp rx=0 tx=0).stop 2024-03-18T15:43:25.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 msgr2=0x7fcc941a5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:25.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 0x7fcc941a5750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00cab0 tx=0x7fcc8c00cf70 comp rx=0 tx=0).stop 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 shutdown_connections 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcc800a2a00 0x7fcc800a4ec0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc941053b0 0x7fcc941a5750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc94102a00 0x7fcc9419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 --2- 172.21.15.96:0/281946418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc94100090 0x7fcc9419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 >> 172.21.15.96:0/281946418 conn(0x7fcc940fbc90 msgr2=0x7fcc941009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 shutdown_connections 2024-03-18T15:43:25.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:25.866+0000 7fcc9b987640 1 -- 172.21.15.96:0/281946418 wait complete. 2024-03-18T15:43:25.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:25 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:43:25.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:25 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:43:25.951 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:43:25.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:43:26.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:25 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:43:26.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:25 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:43:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:25 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:43:26.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:25 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 -- 172.21.15.96:0/2813429926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d81029e0 msgr2=0x7fb2d8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2813429926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d81029e0 0x7fb2d8104e50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009900 tx=0x7fb2cc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 -- 172.21.15.96:0/2813429926 shutdown_connections 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2813429926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8105390 0x7fb2d8105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2813429926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d81029e0 0x7fb2d8104e50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2813429926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d8100070 0x7fb2d81024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 -- 172.21.15.96:0/2813429926 >> 172.21.15.96:0/2813429926 conn(0x7fb2d80fbc70 msgr2=0x7fb2d80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 -- 172.21.15.96:0/2813429926 shutdown_connections 2024-03-18T15:43:26.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 -- 172.21.15.96:0/2813429926 wait complete. 2024-03-18T15:43:26.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.421+0000 7fb2dd337640 1 Processor -- start 2024-03-18T15:43:26.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 -- start start 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d81029e0 0x7fb2d819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 0x7fb2d81a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8113aa0 con 0x7fb2d8105390 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8113920 con 0x7fb2d8100070 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2dd337640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d8113c20 con 0x7fb2d81029e0 2024-03-18T15:43:26.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 0x7fb2d81a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d81029e0 0x7fb2d819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41902/0 (socket says 172.21.15.96:41902) 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 -- 172.21.15.96:0/2522749034 learned_addr learned my addr 172.21.15.96:0/2522749034 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 -- 172.21.15.96:0/2522749034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d81029e0 msgr2=0x7fb2d819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d81029e0 0x7fb2d819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 -- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 msgr2=0x7fb2d81a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 0x7fb2d81a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.422+0000 7fb2d6ffd640 1 -- 172.21.15.96:0/2522749034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d81a5d00 con 0x7fb2d8100070 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2d77fe640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 0x7fb2d81a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2d6ffd640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800e9f0 tx=0x7fb2c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800cd80 con 0x7fb2d8100070 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d81a5ff0 con 0x7fb2d8100070 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8010910 con 0x7fb2d8100070 2024-03-18T15:43:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80188f0 con 0x7fb2d8100070 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.423+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d81a6460 con 0x7fb2d8100070 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.424+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d80fd760 con 0x7fb2d8100070 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.426+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb2c8010ac0 con 0x7fb2d8100070 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.427+0000 7fb2bffff640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 0x7fb2c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.427+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb2c80cefb0 con 0x7fb2d8100070 2024-03-18T15:43:26.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.427+0000 7fb2d67fc640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 0x7fb2c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.427+0000 7fb2d67fc640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 0x7fb2c00a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc02c240 tx=0x7fb2cc02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.429+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb2c809b9d0 con 0x7fb2d8100070 2024-03-18T15:43:26.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.687+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2d802d250 con 0x7fb2d8100070 2024-03-18T15:43:26.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.688+0000 7fb2bffff640 1 -- 172.21.15.96:0/2522749034 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11940 (secure 0 0 0) 0x7fb2c809fa70 con 0x7fb2d8100070 2024-03-18T15:43:26.689 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:43:26.689 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:43:19.439284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:43:26.689 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 msgr2=0x7fb2c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 0x7fb2c00a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc02c240 tx=0x7fb2cc02df90 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 msgr2=0x7fb2d819e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800e9f0 tx=0x7fb2c800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 shutdown_connections 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb2c00a2800 0x7fb2c00a4cc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2d8105390 0x7fb2d81a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d81029e0 0x7fb2d819e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 --2- 172.21.15.96:0/2522749034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d8100070 0x7fb2d819e0e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 >> 172.21.15.96:0/2522749034 conn(0x7fb2d80fbc70 msgr2=0x7fb2d81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 shutdown_connections 2024-03-18T15:43:26.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.690+0000 7fb2dd337640 1 -- 172.21.15.96:0/2522749034 wait complete. 2024-03-18T15:43:26.702 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:43:26.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- 172.21.15.96:0/3985483649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41006e0 msgr2=0x7fdeb4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/3985483649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41006e0 0x7fdeb4100b00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009900 tx=0x7fdeb002bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:26.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- 172.21.15.96:0/3985483649 shutdown_connections 2024-03-18T15:43:26.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/3985483649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb4102290 0x7fdeb4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/3985483649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/3985483649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb41006e0 0x7fdeb4100b00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- 172.21.15.96:0/3985483649 >> 172.21.15.96:0/3985483649 conn(0x7fdeb40fbc70 msgr2=0x7fdeb40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- 172.21.15.96:0/3985483649 shutdown_connections 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- 172.21.15.96:0/3985483649 wait complete. 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 Processor -- start 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- start start 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb41006e0 0x7fdeb4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d440 con 0x7fdeb4102290 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d2c0 con 0x7fdeb41018d0 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.810+0000 7fdebb7d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb410d5c0 con 0x7fdeb41006e0 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39962/0 (socket says 172.21.15.96:39962) 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 -- 172.21.15.96:0/942457030 learned_addr learned my addr 172.21.15.96:0/942457030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9570640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb41006e0 0x7fdeb4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 -- 172.21.15.96:0/942457030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb41006e0 msgr2=0x7fdeb4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb41006e0 0x7fdeb4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 -- 172.21.15.96:0/942457030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 msgr2=0x7fdeb41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 -- 172.21.15.96:0/942457030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb419f7a0 con 0x7fdeb4102290 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeb9d71640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/942457030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac017070 con 0x7fdeb4102290 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb419fa90 con 0x7fdeb4102290 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/942457030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeac003e50 con 0x7fdeb4102290 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/942457030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0106c0 con 0x7fdeb4102290 2024-03-18T15:43:26.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.811+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb419ffe0 con 0x7fdeb4102290 2024-03-18T15:43:26.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.813+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/942457030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fdeac003970 con 0x7fdeb4102290 2024-03-18T15:43:26.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 Processor -- start 2024-03-18T15:43:26.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 -- start start 2024-03-18T15:43:26.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425e7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556642547490 con 0x5566425e2850 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5566424efc90 con 0x5566425e3240 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556642543530 con 0x5566425e83b0 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeaa7fc640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 0x7fdea00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39976/0 (socket says 172.21.15.96:39976) 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/1953227800 learned_addr learned my addr 172.21.15.96:0/1953227800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb9570640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425e7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb9d71640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 0x7fdea00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.814+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeaa7fc640 1 -- 172.21.15.96:0/942457030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdeac0ce8a0 con 0x7fdeb4102290 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/1953227800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 msgr2=0x5566425ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/1953227800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 msgr2=0x5566425e7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425e7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/1953227800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566425aa500 con 0x5566425e2850 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb9570640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb9d71640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 0x7fdea00a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b760 tx=0x7fdeac06dee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdea4013780 tx=0x7fdea4009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34aef988c445f5f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fde9f7fe640 1 -- 172.21.15.96:0/1953227800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4013dc0 con 0x5566425e2850 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fde9f7fe640 1 -- 172.21.15.96:0/1953227800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea40110d0 con 0x5566425e2850 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.815+0000 7fde9f7fe640 1 -- 172.21.15.96:0/1953227800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea400fa70 con 0x5566425e2850 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- 172.21.15.96:0/1953227800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 msgr2=0x5566425e2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdea4013780 tx=0x7fdea4009730 comp rx=0 tx=0).stop 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- 172.21.15.96:0/1953227800 shutdown_connections 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425e7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/1953227800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e2c70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- 172.21.15.96:0/1953227800 >> 172.21.15.96:0/1953227800 conn(0x5566425dddc0 msgr2=0x5566425de1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- 172.21.15.96:0/1953227800 shutdown_connections 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- 172.21.15.96:0/1953227800 wait complete. 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 -- start start 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.816+0000 7fdebe3c2740 1 Processor -- start 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425f4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5566425eb300 con 0x5566425e2850 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556642503230 con 0x5566425e3240 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5566425eb480 con 0x5566425e83b0 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 10 client.0 _ll_register_callbacks cb 0x55664253f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb9570640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39992/0 (socket says 172.21.15.96:39992) 2024-03-18T15:43:26.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/3101712152 learned_addr learned my addr 172.21.15.96:0/3101712152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/3101712152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 msgr2=0x5566425f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 msgr2=0x5566425ed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425ed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/3101712152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556642512100 con 0x5566425e2850 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb9570640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdea4006010 tx=0x7fdea4009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1e62572d1b924a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fde9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4004260 con 0x5566425e2850 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdea4004400 con 0x5566425e2850 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.817+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea40110d0 con 0x5566425e2850 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdea400f7f0 con 0x5566425e2850 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 10 client.16428 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:26.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55664225b3f0 con 0x5566425e2850 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea40110d0 con 0x5566425e2850 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fdea4011b30 con 0x5566425e2850 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 10 client.16428 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 10 client.16428 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 20 client.16428 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425f88a0 0x5566425face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 4 client.16428 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566425549d0 con 0x5566425f88a0 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.818+0000 7fdeb9570640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425f88a0 0x5566425face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fde9cff9640 10 client.16428 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdeb9570640 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425f88a0 0x5566425face0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45b5c3602988fbb server_cookie=b538086e8e85c284 in_seq=0 out_seq=0 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fde9cff9640 1 -- 172.21.15.96:0/3101712152 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5566425549d0 con 0x5566425f88a0 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fde9cff9640 10 client.16428 handle_command_reply: tid=0 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 client.16428 shutdown 2024-03-18T15:43:26.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 msgr2=0x5566425ecd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdea4006010 tx=0x7fdea4009280 comp rx=0 tx=0).stop 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 shutdown_connections 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425f88a0 0x5566425face0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e83b0 0x5566425f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/3101712152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425ecd20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fde9cff9640 0 client.16428 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.819+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 >> 172.21.15.96:0/3101712152 conn(0x5566425dddc0 msgr2=0x5566425dcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.820+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 shutdown_connections 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.820+0000 7fdebe3c2740 1 -- 172.21.15.96:0/3101712152 wait complete. 2024-03-18T15:43:26.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.820+0000 7fdebe3c2740 20 client.16428 trim_cache size 0 max 16384 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 Processor -- start 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 -- start start 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425dddc0 0x5566425e4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a8dc0 con 0x5566425e2850 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41aa990 con 0x5566425e3240 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a9490 con 0x5566425dddc0 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39998/0 (socket says 172.21.15.96:39998) 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 -- 172.21.15.96:0/2984433976 learned_addr learned my addr 172.21.15.96:0/2984433976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425dddc0 0x5566425e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 -- 172.21.15.96:0/2984433976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425dddc0 msgr2=0x5566425e4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425dddc0 0x5566425e4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 -- 172.21.15.96:0/2984433976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 msgr2=0x5566425f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 -- 172.21.15.96:0/2984433976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566423257d0 con 0x5566425e2850 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002cfd0 tx=0x7fdeb002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d421ab740c7b559d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fde9effd640 1 -- 172.21.15.96:0/2984433976 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0004410 con 0x5566425e2850 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fde9effd640 1 -- 172.21.15.96:0/2984433976 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb00462c0 con 0x5566425e2850 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.833+0000 7fde9effd640 1 -- 172.21.15.96:0/2984433976 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0005330 con 0x5566425e2850 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2984433976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 msgr2=0x5566425e4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdeb002cfd0 tx=0x7fdeb002e040 comp rx=0 tx=0).stop 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2984433976 shutdown_connections 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425e3240 0x5566425f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e2850 0x5566425e4c80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2984433976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425dddc0 0x5566425e4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2984433976 >> 172.21.15.96:0/2984433976 conn(0x5566425e4020 msgr2=0x5566425de1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2984433976 shutdown_connections 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2984433976 wait complete. 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 -- start start 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.834+0000 7fdebe3c2740 1 Processor -- start 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 0x5566425f1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 0x5566425f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 0x55664265cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41aa2c0 con 0x5566425e3240 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a9bf0 con 0x5566425dddc0 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb41a6da0 con 0x5566425e2850 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdebe3c2740 10 client.0 _ll_register_callbacks cb 0x7fdeb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 0x55664265cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 0x5566425f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 0x5566425f1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41958/0 (socket says 172.21.15.96:41958) 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb9570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 0x5566425f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/2062532391 learned_addr learned my addr 172.21.15.96:0/2062532391 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/2062532391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 msgr2=0x5566425f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 0x5566425f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 msgr2=0x5566425f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 0x5566425f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 -- 172.21.15.96:0/2062532391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664265d160 con 0x5566425e3240 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 0x5566425f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fdeb8d6f640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 0x55664265cc20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdea4004970 tx=0x7fdea4026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565e3c9ab04234ff server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fde9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.835+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4013600 con 0x5566425e3240 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdea402a040 con 0x5566425e3240 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb00462c0 con 0x5566425e3240 2024-03-18T15:43:26.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdea4040050 con 0x5566425e3240 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 10 client.16434 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55664265e320 con 0x5566425e3240 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb00462c0 con 0x5566425e3240 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fdea400fab0 con 0x5566425e3240 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 10 client.16434 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 10 client.16434 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 20 client.16434 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425e83b0 0x556642660c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 4 client.16434 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55664254ed20 con 0x5566425e83b0 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.836+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425e83b0 0x556642660c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdeb9570640 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425e83b0 0x556642660c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d59bba2cef4c439b server_cookie=84f3a8339d834d4e in_seq=0 out_seq=0 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fde9cff9640 10 client.16434 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fde9cff9640 1 -- 172.21.15.96:0/2062532391 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55664254ed20 con 0x5566425e83b0 2024-03-18T15:43:26.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fde9cff9640 10 client.16434 handle_command_reply: tid=0 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 client.16434 shutdown 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 msgr2=0x55664265cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 0x55664265cc20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdea4004970 tx=0x7fdea4026040 comp rx=0 tx=0).stop 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 shutdown_connections 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x5566425e83b0 0x556642660c90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5566425e3240 0x55664265cc20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566425e2850 0x5566425f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fde9cff9640 0 client.16434 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 --2- 172.21.15.96:0/2062532391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566425dddc0 0x5566425f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 >> 172.21.15.96:0/2062532391 conn(0x5566425e4020 msgr2=0x5566425eccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 shutdown_connections 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 1 -- 172.21.15.96:0/2062532391 wait complete. 2024-03-18T15:43:26.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.837+0000 7fdebe3c2740 20 client.16434 trim_cache size 0 max 16384 2024-03-18T15:43:26.845 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:43:26.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 msgr2=0x7fdea00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 0x7fdea00a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b760 tx=0x7fdeac06dee0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 msgr2=0x7fdeb419f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009fd0 tx=0x7fdeac00b260 comp rx=0 tx=0).stop 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 shutdown_connections 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdea00a2800 0x7fdea00a4cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeb4102290 0x7fdeb419f130 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdeb41018d0 0x7fdeb41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 --2- 172.21.15.96:0/942457030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdeb41006e0 0x7fdeb4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 >> 172.21.15.96:0/942457030 conn(0x7fdeb40fbc70 msgr2=0x7fdeb4104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 shutdown_connections 2024-03-18T15:43:26.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.838+0000 7fdebb7d4640 1 -- 172.21.15.96:0/942457030 wait complete. 2024-03-18T15:43:26.849 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:43:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:26 smithi096 ceph-mon[31930]: pgmap v452: 97 pgs: 97 active+clean; 372 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 -- 172.21.15.96:0/4121017721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 msgr2=0x7faa48101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 --2- 172.21.15.96:0/4121017721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48101d50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009900 tx=0x7faa3c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 -- 172.21.15.96:0/4121017721 shutdown_connections 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 --2- 172.21.15.96:0/4121017721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa48102290 0x7faa48108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 --2- 172.21.15.96:0/4121017721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48101d50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 --2- 172.21.15.96:0/4121017721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa481006e0 0x7faa48100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 -- 172.21.15.96:0/4121017721 >> 172.21.15.96:0/4121017721 conn(0x7faa480fbc70 msgr2=0x7faa480fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 -- 172.21.15.96:0/4121017721 shutdown_connections 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.955+0000 7faa4ed77640 1 -- 172.21.15.96:0/4121017721 wait complete. 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 Processor -- start 2024-03-18T15:43:26.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 -- start start 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa481006e0 0x7faa48197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa48102290 0x7faa4819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d480 con 0x7faa481018d0 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d300 con 0x7faa481006e0 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4ed77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810d600 con 0x7faa48102290 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40016/0 (socket says 172.21.15.96:40016) 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa47fff640 1 -- 172.21.15.96:0/3775193349 learned_addr learned my addr 172.21.15.96:0/3775193349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4cb13640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa481006e0 0x7faa48197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.956+0000 7faa4d314640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa48102290 0x7faa4819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 -- 172.21.15.96:0/3775193349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa48102290 msgr2=0x7faa4819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa48102290 0x7faa4819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 -- 172.21.15.96:0/3775193349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa481006e0 msgr2=0x7faa48197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa481006e0 0x7faa48197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 -- 172.21.15.96:0/3775193349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4819f740 con 0x7faa481018d0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa47fff640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009230 tx=0x7faa3c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa45ffb640 1 -- 172.21.15.96:0/3775193349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c041070 con 0x7faa481018d0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa45ffb640 1 -- 172.21.15.96:0/3775193349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3c0054e0 con 0x7faa481018d0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4819f9d0 con 0x7faa481018d0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa45ffb640 1 -- 172.21.15.96:0/3775193349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c034440 con 0x7faa481018d0 2024-03-18T15:43:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.957+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4819fec0 con 0x7faa481018d0 2024-03-18T15:43:26.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.959+0000 7faa45ffb640 1 -- 172.21.15.96:0/3775193349 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7faa3c044070 con 0x7faa481018d0 2024-03-18T15:43:26.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 Processor -- start 2024-03-18T15:43:26.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 -- start start 2024-03-18T15:43:26.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d662ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6632840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9d658f490 con 0x55b9d66303b0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6537c90 con 0x55b9d662b240 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa51966740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b9d658b530 con 0x55b9d662a850 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa45ffb640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 0x7faa300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4d314640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d662ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6632840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4d314640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 0x7faa300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41994/0 (socket says 172.21.15.96:41994) 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 -- 172.21.15.96:0/1309175836 learned_addr learned my addr 172.21.15.96:0/1309175836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa45ffb640 1 -- 172.21.15.96:0/3775193349 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa3c0f06f0 con 0x7faa481018d0 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 -- 172.21.15.96:0/1309175836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 msgr2=0x55b9d662ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d662ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 -- 172.21.15.96:0/1309175836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 msgr2=0x55b9d6632840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6632840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.960+0000 7faa4cb13640 1 -- 172.21.15.96:0/1309175836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d65f2500 con 0x55b9d662b240 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa4d314640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 0x7faa300a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55b9d662baa0 tx=0x7faa400096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa4cb13640 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa3800b7d0 tx=0x7faa38004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577bb9bb19cfc163 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa2effd640 1 -- 172.21.15.96:0/1309175836 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38003dd0 con 0x55b9d662b240 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa2effd640 1 -- 172.21.15.96:0/1309175836 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa38015070 con 0x55b9d662b240 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa2effd640 1 -- 172.21.15.96:0/1309175836 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38011730 con 0x55b9d662b240 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 -- 172.21.15.96:0/1309175836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 msgr2=0x55b9d662fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faa3800b7d0 tx=0x7faa38004610 comp rx=0 tx=0).stop 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 -- 172.21.15.96:0/1309175836 shutdown_connections 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6632840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d662fd40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.961+0000 7faa51966740 1 --2- 172.21.15.96:0/1309175836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d662ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- 172.21.15.96:0/1309175836 >> 172.21.15.96:0/1309175836 conn(0x55b9d6625dc0 msgr2=0x55b9d66261c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- 172.21.15.96:0/1309175836 shutdown_connections 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- 172.21.15.96:0/1309175836 wait complete. 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- start start 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 Processor -- start 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d66352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d66303b0 0x55b9d663c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9d6633360 con 0x55b9d662a850 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b9d654b230 con 0x55b9d662b240 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b9d66334e0 con 0x55b9d66303b0 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.962+0000 7faa51966740 10 client.0 _ll_register_callbacks cb 0x55b9d6587010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa47fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d66352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40030/0 (socket says 172.21.15.96:40030) 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 -- 172.21.15.96:0/126859239 learned_addr learned my addr 172.21.15.96:0/126859239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4d314640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d66303b0 0x55b9d663c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 -- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d66303b0 msgr2=0x55b9d663c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d66303b0 0x55b9d663c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 -- 172.21.15.96:0/126859239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 msgr2=0x55b9d66352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d66352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 -- 172.21.15.96:0/126859239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d655a100 con 0x55b9d662a850 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa47fff640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d66352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa4cb13640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa38003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e69cd9acecd16d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa27fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3801a880 con 0x55b9d662a850 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa3801aa20 con 0x55b9d662a850 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa38015070 con 0x55b9d662a850 2024-03-18T15:43:26.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa3802d030 con 0x55b9d662a850 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa51966740 10 client.16446 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.963+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9d62a33f0 con 0x55b9d662a850 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa38015070 con 0x55b9d662a850 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faa38027470 con 0x55b9d662a850 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 10 client.16446 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 10 client.16446 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 20 client.16446 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d6640910 0x55b9d6642d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 4 client.16446 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d659c9d0 con 0x55b9d6640910 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa47fff640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d6640910 0x55b9d6642d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa27fff640 10 client.16446 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.964+0000 7faa47fff640 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d6640910 0x55b9d6642d00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4842e0a59c4af10f server_cookie=d94c1d2542ee0cb9 in_seq=0 out_seq=0 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa27fff640 1 -- 172.21.15.96:0/126859239 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b9d659c9d0 con 0x55b9d6640910 2024-03-18T15:43:26.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa27fff640 10 client.16446 handle_command_reply: tid=0 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 client.16446 shutdown 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 msgr2=0x55b9d6634d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faa38000c00 tx=0x7faa38003e60 comp rx=0 tx=0).stop 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 shutdown_connections 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d6640910 0x55b9d6642d00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d66303b0 0x55b9d663c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d66352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 --2- 172.21.15.96:0/126859239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662a850 0x55b9d6634d90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa27fff640 0 client.16446 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.965+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 >> 172.21.15.96:0/126859239 conn(0x55b9d6625dc0 msgr2=0x55b9d6624fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.966+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 shutdown_connections 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.966+0000 7faa51966740 1 -- 172.21.15.96:0/126859239 wait complete. 2024-03-18T15:43:26.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.966+0000 7faa51966740 20 client.16446 trim_cache size 0 max 16384 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 Processor -- start 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 -- start start 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d6629730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d66303b0 0x55b9d6647550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a8d80 con 0x55b9d662b240 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa481aa950 con 0x55b9d66303b0 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa51966740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a9450 con 0x55b9d662a850 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d6629730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40034/0 (socket says 172.21.15.96:40034) 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 -- 172.21.15.96:0/3011644832 learned_addr learned my addr 172.21.15.96:0/3011644832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa4d314640 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d66303b0 0x55b9d6647550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 -- 172.21.15.96:0/3011644832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 msgr2=0x55b9d6629730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d6629730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 -- 172.21.15.96:0/3011644832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d66303b0 msgr2=0x55b9d6647550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d66303b0 0x55b9d6647550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 -- 172.21.15.96:0/3011644832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d636d7d0 con 0x55b9d662b240 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa47fff640 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0b9570 tx=0x7faa3c08fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82485e66b167234 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.979+0000 7faa2e7fc640 1 -- 172.21.15.96:0/3011644832 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c04b040 con 0x55b9d662b240 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa2e7fc640 1 -- 172.21.15.96:0/3011644832 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa3c005690 con 0x55b9d662b240 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa2e7fc640 1 -- 172.21.15.96:0/3011644832 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0c5800 con 0x55b9d662b240 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- 172.21.15.96:0/3011644832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 msgr2=0x55b9d663c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faa3c0b9570 tx=0x7faa3c08fe40 comp rx=0 tx=0).stop 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- 172.21.15.96:0/3011644832 shutdown_connections 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d66303b0 0x55b9d6647550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d662b240 0x55b9d663c2d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 --2- 172.21.15.96:0/3011644832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d6629730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- 172.21.15.96:0/3011644832 >> 172.21.15.96:0/3011644832 conn(0x55b9d6625dc0 msgr2=0x55b9d6629020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- 172.21.15.96:0/3011644832 shutdown_connections 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- 172.21.15.96:0/3011644832 wait complete. 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.980+0000 7faa51966740 1 -- start start 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 Processor -- start 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d663bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d6638af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa481aa280 con 0x55b9d66303b0 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a9bb0 con 0x55b9d662b240 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa481a6d60 con 0x55b9d662a850 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa51966740 10 client.0 _ll_register_callbacks cb 0x7faa3c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4cb13640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d663bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40046/0 (socket says 172.21.15.96:40046) 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 -- 172.21.15.96:0/1166226841 learned_addr learned my addr 172.21.15.96:0/1166226841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa47fff640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d6638af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 -- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 msgr2=0x55b9d663bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d663bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 -- 172.21.15.96:0/1166226841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 msgr2=0x55b9d6638af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d6638af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 -- 172.21.15.96:0/1166226841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d6639620 con 0x55b9d66303b0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.981+0000 7faa4d314640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa4000fbd0 tx=0x7faa40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386e3525350e4a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40030060 con 0x55b9d66303b0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9d662c250 con 0x55b9d66303b0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa40020bd0 con 0x55b9d66303b0 2024-03-18T15:43:26.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa40020dd0 con 0x55b9d66303b0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 10 client.16452 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b9d66a5c60 con 0x55b9d66303b0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b9d662c250 con 0x55b9d66303b0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7faa4002a070 con 0x55b9d66303b0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 10 client.16452 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 10 client.16452 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 20 client.16452 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d66a66c0 0x55b9d66a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 4 client.16452 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b9d6596d20 con 0x55b9d66a66c0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.982+0000 7faa4cb13640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d66a66c0 0x55b9d66a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa4cb13640 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d66a66c0 0x55b9d66a8ae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91bbb308ace17afc server_cookie=9467873a60226a2e in_seq=0 out_seq=0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa27fff640 10 client.16452 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa27fff640 1 -- 172.21.15.96:0/1166226841 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b9d6596d20 con 0x55b9d66a66c0 2024-03-18T15:43:26.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa27fff640 10 client.16452 handle_command_reply: tid=0 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 client.16452 shutdown 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 msgr2=0x55b9d6639030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7faa4000fbd0 tx=0x7faa40004a40 comp rx=0 tx=0).stop 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 shutdown_connections 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x55b9d66a66c0 0x55b9d66a8ae0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9d66303b0 0x55b9d6639030 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9d662b240 0x55b9d6638af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa27fff640 0 client.16452 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 --2- 172.21.15.96:0/1166226841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b9d662a850 0x55b9d663bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 >> 172.21.15.96:0/1166226841 conn(0x55b9d6625dc0 msgr2=0x55b9d66298b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 shutdown_connections 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 1 -- 172.21.15.96:0/1166226841 wait complete. 2024-03-18T15:43:26.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.983+0000 7faa51966740 20 client.16452 trim_cache size 0 max 16384 2024-03-18T15:43:26.991 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:43:26.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 msgr2=0x7faa300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 0x7faa300a4cc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55b9d662baa0 tx=0x7faa400096a0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 msgr2=0x7faa48198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009230 tx=0x7faa3c005dd0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 shutdown_connections 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa300a2800 0x7faa300a4cc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa48102290 0x7faa4819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa481018d0 0x7faa48198060 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 --2- 172.21.15.96:0/3775193349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa481006e0 0x7faa48197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 >> 172.21.15.96:0/3775193349 conn(0x7faa480fbc70 msgr2=0x7faa48104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 shutdown_connections 2024-03-18T15:43:26.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:26.984+0000 7faa4ed77640 1 -- 172.21.15.96:0/3775193349 wait complete. 2024-03-18T15:43:26.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:43:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:26 smithi137 ceph-mon[35868]: pgmap v452: 97 pgs: 97 active+clean; 372 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 -- 172.21.15.96:0/2757086147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a0102290 msgr2=0x7ff2a0108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 --2- 172.21.15.96:0/2757086147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a0102290 0x7ff2a0108b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff298009ca0 tx=0x7ff29802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 -- 172.21.15.96:0/2757086147 shutdown_connections 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 --2- 172.21.15.96:0/2757086147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a0102290 0x7ff2a0108b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 --2- 172.21.15.96:0/2757086147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a01018d0 0x7ff2a0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 --2- 172.21.15.96:0/2757086147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01006e0 0x7ff2a0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 -- 172.21.15.96:0/2757086147 >> 172.21.15.96:0/2757086147 conn(0x7ff2a00fbc70 msgr2=0x7ff2a00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 -- 172.21.15.96:0/2757086147 shutdown_connections 2024-03-18T15:43:27.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.110+0000 7ff2a511b640 1 -- 172.21.15.96:0/2757086147 wait complete. 2024-03-18T15:43:27.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.111+0000 7ff2a511b640 1 Processor -- start 2024-03-18T15:43:27.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.111+0000 7ff2a511b640 1 -- start start 2024-03-18T15:43:27.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 0x7ff2a019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a0102290 0x7ff2a019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010c360 con 0x7ff2a01018d0 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010c1e0 con 0x7ff2a01006e0 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff2a511b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a010c4e0 con 0x7ff2a0102290 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 0x7ff2a019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42054/0 (socket says 172.21.15.96:42054) 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1814458835 learned_addr learned my addr 172.21.15.96:0/1814458835 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 0x7ff2a019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40066/0 (socket says 172.21.15.96:40066) 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29f59e640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a0102290 0x7ff2a019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1814458835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a0102290 msgr2=0x7ff2a019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a0102290 0x7ff2a019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 msgr2=0x7ff2a019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 0x7ff2a019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1814458835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a019f720 con 0x7ff2a01006e0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.112+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff29000e9c0 tx=0x7ff29000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.113+0000 7ff287fff640 1 -- 172.21.15.96:0/1814458835 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29000cd80 con 0x7ff2a01006e0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.113+0000 7ff287fff640 1 -- 172.21.15.96:0/1814458835 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900035f0 con 0x7ff2a01006e0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.113+0000 7ff287fff640 1 -- 172.21.15.96:0/1814458835 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290010690 con 0x7ff2a01006e0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.113+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a019fa10 con 0x7ff2a01006e0 2024-03-18T15:43:27.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.113+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a019ffb0 con 0x7ff2a01006e0 2024-03-18T15:43:27.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.116+0000 7ff287fff640 1 -- 172.21.15.96:0/1814458835 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff290004020 con 0x7ff2a01006e0 2024-03-18T15:43:27.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.116+0000 7ff287fff640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 0x7ff2880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.116+0000 7ff287fff640 1 -- 172.21.15.96:0/1814458835 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff2900cedf0 con 0x7ff2a01006e0 2024-03-18T15:43:27.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.116+0000 7ff29e59c640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 0x7ff2880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 Processor -- start 2024-03-18T15:43:27.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 -- start start 2024-03-18T15:43:27.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb39650 0x55a3aeb3e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29e59c640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 0x7ff2880a4d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0069280 tx=0x7ff294005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a3aea99490 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3aea41c90 con 0x55a3aeb39650 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a3aea95530 con 0x55a3aeb38c60 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb39080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48206/0 (socket says 172.21.15.96:48206) 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40076/0 (socket says 172.21.15.96:40076) 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29ed9d640 1 -- 172.21.15.96:0/2819379046 learned_addr learned my addr 172.21.15.96:0/2819379046 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.117+0000 7ff29e59c640 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb39650 0x55a3aeb3e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 -- 172.21.15.96:0/2819379046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 msgr2=0x55a3aeb39080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb39080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 -- 172.21.15.96:0/2819379046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb39650 msgr2=0x55a3aeb3e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb39650 0x55a3aeb3e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 -- 172.21.15.96:0/2819379046 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3aeafc500 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff298037040 tx=0x7ff2980323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db37c375672711d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff2857fa640 1 -- 172.21.15.96:0/2819379046 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298043070 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff2857fa640 1 -- 172.21.15.96:0/2819379046 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29802cca0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.118+0000 7ff2857fa640 1 -- 172.21.15.96:0/2819379046 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298039040 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2819379046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 msgr2=0x55a3aeb40c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff298037040 tx=0x7ff2980323d0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2819379046 shutdown_connections 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb40c50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb39650 0x55a3aeb3e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2819379046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb39080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2819379046 >> 172.21.15.96:0/2819379046 conn(0x55a3aeb2fdc0 msgr2=0x55a3aeb301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2819379046 shutdown_connections 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.119+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2819379046 wait complete. 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 -- start start 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 Processor -- start 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 0x55a3aeb430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a3aeb41710 con 0x55a3aeb38c60 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 0x55a3aeb430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3aea55230 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a3aeb41890 con 0x55a3aeb39650 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 0x55a3aeb430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40084/0 (socket says 172.21.15.96:40084) 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29ed9d640 1 -- 172.21.15.96:0/2380862677 learned_addr learned my addr 172.21.15.96:0/2380862677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff2a7de3740 10 client.0 _ll_register_callbacks cb 0x55a3aea91010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29e59c640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29f59e640 1 -- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 msgr2=0x55a3aeb435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29f59e640 1 -- 172.21.15.96:0/2380862677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 msgr2=0x55a3aeb430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.120+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 0x55a3aeb430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff29f59e640 1 -- 172.21.15.96:0/2380862677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3aea64100 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff29f59e640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb4a620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff29802c630 tx=0x7ff298003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae5f8c294c7259e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29804c030 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff2980328a0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff29802cca0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff298039070 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff2a7de3740 10 client.26258 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3ae7ad3f0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29802cca0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.121+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff29805a070 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 10 client.26258 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 10 client.26258 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 20 client.26258 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aeb4eb80 0x55a3aeb50ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 4 client.26258 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3aeaa69d0 con 0x55a3aeb4eb80 2024-03-18T15:43:27.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aeb4eb80 0x55a3aeb50ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff282ff5640 10 client.26258 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:43:27.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.122+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aeb4eb80 0x55a3aeb50ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1e0520174ce7adbd server_cookie=3265ad2a19014322 in_seq=0 out_seq=0 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff282ff5640 1 -- 172.21.15.96:0/2380862677 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a3aeaa69d0 con 0x55a3aeb4eb80 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff282ff5640 10 client.26258 handle_command_reply: tid=0 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 client.26258 shutdown 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 msgr2=0x55a3aeb4a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb4a620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff29802c630 tx=0x7ff298003970 comp rx=0 tx=0).stop 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 shutdown_connections 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aeb4eb80 0x55a3aeb50ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb4a620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/2380862677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb38c60 0x55a3aeb430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff282ff5640 0 client.26258 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:43:27.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 >> 172.21.15.96:0/2380862677 conn(0x55a3aeb2fdc0 msgr2=0x55a3aeb2efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.125+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 shutdown_connections 2024-03-18T15:43:27.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.126+0000 7ff2a7de3740 1 -- 172.21.15.96:0/2380862677 wait complete. 2024-03-18T15:43:27.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.126+0000 7ff2a7de3740 20 client.26258 trim_cache size 0 max 16384 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 Processor -- start 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 -- start start 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 0x55a3aeb2ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a6bd0 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a87a0 con 0x55a3aeb38c60 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a72a0 con 0x55a3aeb39650 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb4a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40092/0 (socket says 172.21.15.96:40092) 2024-03-18T15:43:27.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff29f59e640 1 -- 172.21.15.96:0/3488137167 learned_addr learned my addr 172.21.15.96:0/3488137167 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.149+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 0x55a3aeb2ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 -- 172.21.15.96:0/3488137167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 msgr2=0x55a3aeb4a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb4a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 -- 172.21.15.96:0/3488137167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 msgr2=0x55a3aeb2e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 -- 172.21.15.96:0/3488137167 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3aeb2b640 con 0x55a3aeb38c60 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29f59e640 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 0x55a3aeb2ae70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff2900cc910 tx=0x7ff2900cc790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910fd8df0a912dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff284ff9640 1 -- 172.21.15.96:0/3488137167 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29009b760 con 0x55a3aeb38c60 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff284ff9640 1 -- 172.21.15.96:0/3488137167 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900b9070 con 0x55a3aeb38c60 2024-03-18T15:43:27.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.150+0000 7ff284ff9640 1 -- 172.21.15.96:0/3488137167 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290003b30 con 0x55a3aeb38c60 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 -- 172.21.15.96:0/3488137167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 msgr2=0x55a3aeb2ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 0x55a3aeb2ae70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff2900cc910 tx=0x7ff2900cc790 comp rx=0 tx=0).stop 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 -- 172.21.15.96:0/3488137167 shutdown_connections 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb3e7c0 0x55a3aeb2e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb39650 0x55a3aeb4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/3488137167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb38c60 0x55a3aeb2ae70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 -- 172.21.15.96:0/3488137167 >> 172.21.15.96:0/3488137167 conn(0x55a3aeb2fdc0 msgr2=0x55a3aeb430a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 -- 172.21.15.96:0/3488137167 shutdown_connections 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.151+0000 7ff2a7de3740 1 -- 172.21.15.96:0/3488137167 wait complete. 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 -- start start 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 Processor -- start 2024-03-18T15:43:27.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 0x55a3aeb47b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aebb2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a80d0 con 0x55a3aeb39650 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a7a00 con 0x55a3aeb3e7c0 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a01a4bb0 con 0x55a3aeb38c60 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 0x55a3aeb47b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aebb2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48226/0 (socket says 172.21.15.96:48226) 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1331111044 learned_addr learned my addr 172.21.15.96:0/1331111044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.152+0000 7ff2a7de3740 10 client.0 _ll_register_callbacks cb 0x7ff294009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1331111044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 msgr2=0x55a3aebb2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aebb2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1331111044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 msgr2=0x55a3aeb47b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 0x55a3aeb47b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 -- 172.21.15.96:0/1331111044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3ae8777d0 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29f59e640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aebb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29e59c640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 0x55a3aeb47b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff29ed9d640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2900a39d0 tx=0x7ff290099bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=386fde9f8f42b5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff282ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.153+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29007e030 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff290086040 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2900b9070 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff29009f8b0 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 10 client.26317 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3aebb45a0 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2900a3f00 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff2900afb40 con 0x55a3aeb38c60 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 10 client.26317 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 10 client.26317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 20 client.26317 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aebb4f00 0x55a3aebb7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 4 client.26317 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.154+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff2900b9070 con 0x55a3aebb4f00 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.155+0000 7ff29e59c640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aebb4f00 0x55a3aebb7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.155+0000 7ff282ff5640 10 client.26317 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:43:27.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.155+0000 7ff29e59c640 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aebb4f00 0x55a3aebb7320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dbd35eccb4c3a71 server_cookie=728043fe3be0a0c3 in_seq=0 out_seq=0 2024-03-18T15:43:27.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff282ff5640 1 -- 172.21.15.96:0/1331111044 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff2900b9070 con 0x55a3aebb4f00 2024-03-18T15:43:27.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff282ff5640 10 client.26317 handle_command_reply: tid=0 2024-03-18T15:43:27.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 client.26317 shutdown 2024-03-18T15:43:27.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 msgr2=0x55a3aeb47610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2900a39d0 tx=0x7ff290099bb0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 shutdown_connections 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55a3aebb4f00 0x55a3aebb7320 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3aeb3e7c0 0x55a3aebb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a3aeb39650 0x55a3aeb47b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff282ff5640 0 client.26317 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 --2- 172.21.15.96:0/1331111044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a3aeb38c60 0x55a3aeb47610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 >> 172.21.15.96:0/1331111044 conn(0x55a3aeb2fdc0 msgr2=0x55a3aeb33020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 shutdown_connections 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 1 -- 172.21.15.96:0/1331111044 wait complete. 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.158+0000 7ff2a7de3740 20 client.26317 trim_cache size 0 max 16384 2024-03-18T15:43:27.161 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 msgr2=0x7ff2880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 0x7ff2880a4d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0069280 tx=0x7ff294005f90 comp rx=0 tx=0).stop 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 msgr2=0x7ff2a0069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff29000e9c0 tx=0x7ff29000ee80 comp rx=0 tx=0).stop 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 shutdown_connections 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff2880a2880 0x7ff2880a4d40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2a0102290 0x7ff2a019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2a01018d0 0x7ff2a019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 --2- 172.21.15.96:0/1814458835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a01006e0 0x7ff2a0069510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 >> 172.21.15.96:0/1814458835 conn(0x7ff2a00fbc70 msgr2=0x7ff2a0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 shutdown_connections 2024-03-18T15:43:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.159+0000 7ff2a511b640 1 -- 172.21.15.96:0/1814458835 wait complete. 2024-03-18T15:43:27.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:26 smithi193 ceph-mon[36702]: pgmap v452: 97 pgs: 97 active+clean; 372 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 180 op/s 2024-03-18T15:43:27.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- 172.21.15.96:0/2443819633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 msgr2=0x7f7b88108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 --2- 172.21.15.96:0/2443819633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b88108c00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009ca0 tx=0x7f7b7802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- 172.21.15.96:0/2443819633 shutdown_connections 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 --2- 172.21.15.96:0/2443819633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b88108c00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 --2- 172.21.15.96:0/2443819633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 0x7f7b88100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 --2- 172.21.15.96:0/2443819633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b88102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- 172.21.15.96:0/2443819633 >> 172.21.15.96:0/2443819633 conn(0x7f7b880fb3d0 msgr2=0x7f7b880fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- 172.21.15.96:0/2443819633 shutdown_connections 2024-03-18T15:43:27.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- 172.21.15.96:0/2443819633 wait complete. 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 Processor -- start 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.307+0000 7f7b8d863640 1 -- start start 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 0x7f7b88199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b881a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810c370 con 0x7f7b88101360 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810c1f0 con 0x7f7b88101c70 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8d863640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8810c4f0 con 0x7f7b881009a0 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 0x7f7b88199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b881a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40122/0 (socket says 172.21.15.96:40122) 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 -- 172.21.15.96:0/4011186605 learned_addr learned my addr 172.21.15.96:0/4011186605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 -- 172.21.15.96:0/4011186605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 msgr2=0x7f7b88199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 0x7f7b88199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 -- 172.21.15.96:0/4011186605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 msgr2=0x7f7b881a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b881a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 -- 172.21.15.96:0/4011186605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881a1910 con 0x7f7b88101360 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b877fe640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b881a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000e9f0 tx=0x7f7b7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.308+0000 7f7b8c861640 1 -- 172.21.15.96:0/4011186605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000cd80 con 0x7f7b88101360 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.309+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a1c00 con 0x7f7b88101360 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.309+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a2150 con 0x7f7b88101360 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.309+0000 7f7b8c861640 1 -- 172.21.15.96:0/4011186605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b70005500 con 0x7f7b88101360 2024-03-18T15:43:27.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.309+0000 7f7b8c861640 1 -- 172.21.15.96:0/4011186605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70010690 con 0x7f7b88101360 2024-03-18T15:43:27.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.311+0000 7f7b8c861640 1 -- 172.21.15.96:0/4011186605 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7b70010830 con 0x7f7b88101360 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.311+0000 7f7b90446740 1 Processor -- start 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.311+0000 7f7b90446740 1 -- start start 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf93850 0x55975bf93c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf993b0 0x55975bf9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55975bef8490 con 0x55975bf94240 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55975bea0c90 con 0x55975bf93850 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b90446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55975bef4530 con 0x55975bf993b0 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b8c861640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 0x7f7b740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40134/0 (socket says 172.21.15.96:40134) 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 -- 172.21.15.96:0/1852378505 learned_addr learned my addr 172.21.15.96:0/1852378505 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b867fc640 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf993b0 0x55975bf9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b877fe640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 0x7f7b740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b877fe640 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf93850 0x55975bf93c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b8c861640 1 -- 172.21.15.96:0/4011186605 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b700cef20 con 0x7f7b88101360 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 -- 172.21.15.96:0/1852378505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf993b0 msgr2=0x55975bf9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf993b0 0x55975bf9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 -- 172.21.15.96:0/1852378505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf93850 msgr2=0x55975bf93c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf93850 0x55975bf93c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 -- 172.21.15.96:0/1852378505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975bf5b500 con 0x55975bf94240 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.312+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009c60 tx=0x7f7b80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7f8bda7fa9e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b7dffb640 1 -- 172.21.15.96:0/1852378505 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004220 con 0x55975bf94240 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b877fe640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 0x7f7b740a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802c630 tx=0x7f7b7802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b7dffb640 1 -- 172.21.15.96:0/1852378505 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b800043c0 con 0x55975bf94240 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b7dffb640 1 -- 172.21.15.96:0/1852378505 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80004d50 con 0x55975bf94240 2024-03-18T15:43:27.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b90446740 1 -- 172.21.15.96:0/1852378505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 msgr2=0x55975bf98d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b90446740 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009c60 tx=0x7f7b80011810 comp rx=0 tx=0).stop 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.313+0000 7f7b90446740 1 -- 172.21.15.96:0/1852378505 shutdown_connections 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf993b0 0x55975bf9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf94240 0x55975bf98d40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 --2- 172.21.15.96:0/1852378505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf93850 0x55975bf93c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 -- 172.21.15.96:0/1852378505 >> 172.21.15.96:0/1852378505 conn(0x55975bf8edc0 msgr2=0x55975bf8f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 -- 172.21.15.96:0/1852378505 shutdown_connections 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 -- 172.21.15.96:0/1852378505 wait complete. 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 -- start start 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 Processor -- start 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.314+0000 7f7b90446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf9dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bf9e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bfa5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55975bf9c360 con 0x55975bf993b0 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55975beb4230 con 0x55975bf94240 2024-03-18T15:43:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55975bf9c4e0 con 0x55975bf93850 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b90446740 10 client.0 _ll_register_callbacks cb 0x55975bef0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf9dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf9dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48260/0 (socket says 172.21.15.96:48260) 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bf9e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b86ffd640 1 -- 172.21.15.96:0/1211281793 learned_addr learned my addr 172.21.15.96:0/1211281793 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bfa5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 -- 172.21.15.96:0/1211281793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 msgr2=0x55975bf9dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf9dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 -- 172.21.15.96:0/1211281793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 msgr2=0x55975bfa5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bfa5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 -- 172.21.15.96:0/1211281793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975bec3100 con 0x55975bf94240 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b867fc640 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bf9e270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b7009b590 tx=0x7f7b70011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0aa4a4fb542ebe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.315+0000 7f7b66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700999d0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b700860b0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b800043c0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b700b4050 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 10 client.26276 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975bc0c3f0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b800043c0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f7b700ab5c0 con 0x55975bf94240 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 10 client.26276 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 10 client.26276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 20 client.26276 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975bfabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 4 client.26276 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.316+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55975bf059d0 con 0x55975bfa9b90 2024-03-18T15:43:27.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.317+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975bfabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.317+0000 7f7b66ffd640 10 client.26276 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:43:27.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.317+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975bfabfe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6b5032270f15eab server_cookie=517c41f3a3ac0511 in_seq=0 out_seq=0 2024-03-18T15:43:27.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b66ffd640 1 -- 172.21.15.96:0/1211281793 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55975bf059d0 con 0x55975bfa9b90 2024-03-18T15:43:27.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b66ffd640 10 client.26276 handle_command_reply: tid=0 2024-03-18T15:43:27.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 client.26276 shutdown 2024-03-18T15:43:27.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 msgr2=0x55975bf9e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bf9e270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b7009b590 tx=0x7f7b70011000 comp rx=0 tx=0).stop 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 shutdown_connections 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bfa5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975bfabfe0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bf9e270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b66ffd640 0 client.26276 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 --2- 172.21.15.96:0/1211281793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf9dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 >> 172.21.15.96:0/1211281793 conn(0x55975bf8edc0 msgr2=0x55975bf8dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 shutdown_connections 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 1 -- 172.21.15.96:0/1211281793 wait complete. 2024-03-18T15:43:27.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.318+0000 7f7b90446740 20 client.26276 trim_cache size 0 max 16384 2024-03-18T15:43:27.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 Processor -- start 2024-03-18T15:43:27.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 -- start start 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881aaeb0 con 0x55975bf993b0 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881aca80 con 0x55975bf94240 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b90446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ab580 con 0x55975bf93850 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42116/0 (socket says 172.21.15.96:42116) 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b877fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40146/0 (socket says 172.21.15.96:40146) 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 -- 172.21.15.96:0/4107469054 learned_addr learned my addr 172.21.15.96:0/4107469054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 -- 172.21.15.96:0/4107469054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 msgr2=0x55975bf8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 -- 172.21.15.96:0/4107469054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 msgr2=0x55975bf8dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 -- 172.21.15.96:0/4107469054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975bcd67d0 con 0x55975bf94240 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b877fe640 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.332+0000 7f7b867fc640 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b70097c60 tx=0x7f7b70096200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f805a544954c27c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b7d7fa640 1 -- 172.21.15.96:0/4107469054 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700b8070 con 0x55975bf94240 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b7d7fa640 1 -- 172.21.15.96:0/4107469054 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b700a3e20 con 0x55975bf94240 2024-03-18T15:43:27.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b7d7fa640 1 -- 172.21.15.96:0/4107469054 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7009b890 con 0x55975bf94240 2024-03-18T15:43:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 -- 172.21.15.96:0/4107469054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 msgr2=0x55975bfa5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b70097c60 tx=0x7f7b70096200 comp rx=0 tx=0).stop 2024-03-18T15:43:27.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 -- 172.21.15.96:0/4107469054 shutdown_connections 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf993b0 0x55975bf8dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf94240 0x55975bfa5140 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 --2- 172.21.15.96:0/4107469054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf93850 0x55975bf8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 -- 172.21.15.96:0/4107469054 >> 172.21.15.96:0/4107469054 conn(0x55975bf8edc0 msgr2=0x55975bf8f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 -- 172.21.15.96:0/4107469054 shutdown_connections 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.333+0000 7f7b90446740 1 -- 172.21.15.96:0/4107469054 wait complete. 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 -- start start 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 Processor -- start 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 0x55975bfa33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 0x55975bfa43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881ac3b0 con 0x55975bf93850 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881abce0 con 0x55975bf993b0 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881a8e90 con 0x55975bf94240 2024-03-18T15:43:27.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b90446740 10 client.0 _ll_register_callbacks cb 0x7f7b70097aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40150/0 (socket says 172.21.15.96:40150) 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b867fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 0x55975bfa33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b86ffd640 1 -- 172.21.15.96:0/3573514920 learned_addr learned my addr 172.21.15.96:0/3573514920 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.334+0000 7f7b877fe640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 0x55975bfa43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 -- 172.21.15.96:0/3573514920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 msgr2=0x55975bfa33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 0x55975bfa33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 -- 172.21.15.96:0/3573514920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 msgr2=0x55975bfa2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 -- 172.21.15.96:0/3573514920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975bfa49b0 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b867fc640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 0x55975bfa33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b877fe640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 0x55975bfa43c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b78037040 tx=0x7f7b7802c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d726072126a7a15d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b78053070 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b78034c00 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b700a3e20 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7b78039040 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b90446740 10 client.26282 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55975c010380 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.335+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b700a3e20 con 0x55975bf993b0 2024-03-18T15:43:27.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f7b78061020 con 0x55975bf993b0 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 10 client.26282 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 10 client.26282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 20 client.26282 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975c012d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 4 client.26282 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55975beffd20 con 0x55975bfa9b90 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975c012d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b66ffd640 10 client.26282 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.336+0000 7f7b86ffd640 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975c012d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=638280508b63f716 server_cookie=82b57a3d2604de40 in_seq=0 out_seq=0 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b66ffd640 1 -- 172.21.15.96:0/3573514920 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55975beffd20 con 0x55975bfa9b90 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b66ffd640 10 client.26282 handle_command_reply: tid=0 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 client.26282 shutdown 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 msgr2=0x55975bfa43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 0x55975bfa43c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7b78037040 tx=0x7f7b7802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 shutdown_connections 2024-03-18T15:43:27.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55975bfa9b90 0x55975c012d60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55975bf993b0 0x55975bfa43c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55975bf94240 0x55975bfa33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b66ffd640 0 client.26282 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 --2- 172.21.15.96:0/3573514920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55975bf93850 0x55975bfa2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 >> 172.21.15.96:0/3573514920 conn(0x55975bf8edc0 msgr2=0x55975bf8d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 shutdown_connections 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 1 -- 172.21.15.96:0/3573514920 wait complete. 2024-03-18T15:43:27.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.337+0000 7f7b90446740 20 client.26282 trim_cache size 0 max 16384 2024-03-18T15:43:27.345 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:43:27.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 msgr2=0x7f7b740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 0x7f7b740a4cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802c630 tx=0x7f7b7802e040 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 msgr2=0x7f7b8819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000e9f0 tx=0x7f7b7000edf0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 shutdown_connections 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b740a2800 0x7f7b740a4cc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b88101c70 0x7f7b881a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b88101360 0x7f7b8819a240 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 --2- 172.21.15.96:0/4011186605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b881009a0 0x7f7b88199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 >> 172.21.15.96:0/4011186605 conn(0x7f7b880fb3d0 msgr2=0x7f7b880fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 shutdown_connections 2024-03-18T15:43:27.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.338+0000 7f7b8d863640 1 -- 172.21.15.96:0/4011186605 wait complete. 2024-03-18T15:43:27.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:43:27.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.473+0000 7f32a85b9640 1 -- 172.21.15.96:0/455503698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a0102290 msgr2=0x7f32a0108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.473+0000 7f32a85b9640 1 --2- 172.21.15.96:0/455503698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a0102290 0x7f32a0108b20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f329c009900 tx=0x7f329c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- 172.21.15.96:0/455503698 shutdown_connections 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- 172.21.15.96:0/455503698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a0102290 0x7f32a0108b20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- 172.21.15.96:0/455503698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a01018d0 0x7f32a0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- 172.21.15.96:0/455503698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01006e0 0x7f32a0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- 172.21.15.96:0/455503698 >> 172.21.15.96:0/455503698 conn(0x7f32a00fbc70 msgr2=0x7f32a00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- 172.21.15.96:0/455503698 shutdown_connections 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- 172.21.15.96:0/455503698 wait complete. 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 Processor -- start 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- start start 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01018d0 0x7f32a0198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 0x7f32a019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32a010c3b0 con 0x7f32a01006e0 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a010c230 con 0x7f32a0102290 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a85b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a010c530 con 0x7f32a01018d0 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a6355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a6355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40168/0 (socket says 172.21.15.96:40168) 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a6355640 1 -- 172.21.15.96:0/3720974544 learned_addr learned my addr 172.21.15.96:0/3720974544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 0x7f32a019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.474+0000 7f32a5b54640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01018d0 0x7f32a0198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 -- 172.21.15.96:0/3720974544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01018d0 msgr2=0x7f32a0198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01018d0 0x7f32a0198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 -- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 msgr2=0x7f32a0197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 -- 172.21.15.96:0/3720974544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a019f6e0 con 0x7f32a0102290 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6355640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 0x7f32a019f070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f329c02c170 tx=0x7f329c02c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32937fe640 1 -- 172.21.15.96:0/3720974544 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c043020 con 0x7f32a0102290 2024-03-18T15:43:27.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32937fe640 1 -- 172.21.15.96:0/3720974544 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c0054e0 con 0x7f32a0102290 2024-03-18T15:43:27.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a019f970 con 0x7f32a0102290 2024-03-18T15:43:27.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32937fe640 1 -- 172.21.15.96:0/3720974544 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c003b40 con 0x7f32a0102290 2024-03-18T15:43:27.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.475+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a019fe90 con 0x7f32a0102290 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.478+0000 7f32937fe640 1 -- 172.21.15.96:0/3720974544 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f329c046020 con 0x7f32a0102290 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.478+0000 7f32ab1a4740 1 Processor -- start 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.478+0000 7f32ab1a4740 1 -- start start 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef5850 0x55cdeaef5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaefad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdeae5a490 con 0x55cdeaefb3b0 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cdeae02c90 con 0x55cdeaef5850 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeae56530 con 0x55cdeaef6240 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a6355640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaefad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a6355640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaefad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48286/0 (socket says 172.21.15.96:48286) 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40172/0 (socket says 172.21.15.96:40172) 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a6355640 1 -- 172.21.15.96:0/971414446 learned_addr learned my addr 172.21.15.96:0/971414446 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef5850 0x55cdeaef5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 -- 172.21.15.96:0/971414446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 msgr2=0x55cdeaefad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaefad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 -- 172.21.15.96:0/971414446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef5850 msgr2=0x55cdeaef5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef5850 0x55cdeaef5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 -- 172.21.15.96:0/971414446 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdeaebd500 con 0x55cdeaefb3b0 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32937fe640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 0x7f328c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a5b54640 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3294009a00 tx=0x7f3294009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9d89aa62ff7d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.479+0000 7f32a6355640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 0x7f328c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f3290bfe640 1 -- 172.21.15.96:0/971414446 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294011070 con 0x55cdeaefb3b0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f3290bfe640 1 -- 172.21.15.96:0/971414446 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294003e50 con 0x55cdeaefb3b0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f3290bfe640 1 -- 172.21.15.96:0/971414446 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940042b0 con 0x55cdeaefb3b0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32937fe640 1 -- 172.21.15.96:0/3720974544 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f329c0f0a20 con 0x7f32a0102290 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 -- 172.21.15.96:0/971414446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 msgr2=0x55cdeaefd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32a6355640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 0x7f328c0a4cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55cdeaef6640 tx=0x7f3298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3294009a00 tx=0x7f3294009990 comp rx=0 tx=0).stop 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 -- 172.21.15.96:0/971414446 shutdown_connections 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefd840 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaefad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/971414446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef5850 0x55cdeaef5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 -- 172.21.15.96:0/971414446 >> 172.21.15.96:0/971414446 conn(0x55cdeaef0dc0 msgr2=0x55cdeaef11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.480+0000 7f32ab1a4740 1 -- 172.21.15.96:0/971414446 shutdown_connections 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 -- 172.21.15.96:0/971414446 wait complete. 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 -- start start 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 Processor -- start 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaf00320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cdeaefe360 con 0x55cdeaef5850 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55cdeae16230 con 0x55cdeaefb3b0 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cdeaefe4e0 con 0x55cdeaef6240 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32ab1a4740 10 client.0 _ll_register_callbacks cb 0x55cdeae52010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40184/0 (socket says 172.21.15.96:40184) 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 -- 172.21.15.96:0/4029248840 learned_addr learned my addr 172.21.15.96:0/4029248840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a5b54640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaf00320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6b56640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 -- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 msgr2=0x55cdeaf00320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaf00320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 -- 172.21.15.96:0/4029248840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaefb3b0 msgr2=0x55cdeaf07320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf07320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.481+0000 7f32a6355640 1 -- 172.21.15.96:0/4029248840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdeae25100 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32a6355640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f329800f920 tx=0x7f3298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b5f45b64a68b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298011940 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3298011ae0 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3294003e50 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3298011ce0 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 10 client.16500 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdeab6e3f0 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294003e50 con 0x55cdeaef5850 2024-03-18T15:43:27.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f3298032070 con 0x55cdeaef5850 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 10 client.16500 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 20 client.16500 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf0bcb0 0x55cdeaf0e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 4 client.16500 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.482+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdeae679d0 con 0x55cdeaf0bcb0 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.483+0000 7f32a5b54640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf0bcb0 0x55cdeaf0e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.483+0000 7f32a5b54640 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf0bcb0 0x55cdeaf0e150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebe4aaf433acd03 server_cookie=6c8ee4d080668d5e in_seq=0 out_seq=0 2024-03-18T15:43:27.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.483+0000 7f3289ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f3289ffb640 1 -- 172.21.15.96:0/4029248840 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55cdeae679d0 con 0x55cdeaf0bcb0 2024-03-18T15:43:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f3289ffb640 10 client.16500 handle_command_reply: tid=0 2024-03-18T15:43:27.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 client.16500 shutdown 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 msgr2=0x55cdeaeffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f329800f920 tx=0x7f3298004730 comp rx=0 tx=0).stop 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 shutdown_connections 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf07320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef6240 0x55cdeaf00320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf0bcb0 0x55cdeaf0e150 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/4029248840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaef5850 0x55cdeaeffde0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f3289ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 >> 172.21.15.96:0/4029248840 conn(0x55cdeaef0dc0 msgr2=0x55cdeaeeffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 shutdown_connections 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 1 -- 172.21.15.96:0/4029248840 wait complete. 2024-03-18T15:43:27.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.484+0000 7f32ab1a4740 20 client.16500 trim_cache size 0 max 16384 2024-03-18T15:43:27.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 Processor -- start 2024-03-18T15:43:27.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 -- start start 2024-03-18T15:43:27.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaefe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01a8d00 con 0x55cdeaefb3b0 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01aa8d0 con 0x55cdeaef6240 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01a93d0 con 0x55cdeaef5850 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a5b54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40190/0 (socket says 172.21.15.96:40190) 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a5b54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42184/0 (socket says 172.21.15.96:42184) 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 -- 172.21.15.96:0/3474809050 learned_addr learned my addr 172.21.15.96:0/3474809050 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6355640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaefe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 -- 172.21.15.96:0/3474809050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 msgr2=0x55cdeaefe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaefe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 -- 172.21.15.96:0/3474809050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 msgr2=0x55cdeaf072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 -- 172.21.15.96:0/3474809050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdeac387d0 con 0x55cdeaefb3b0 2024-03-18T15:43:27.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a5b54640 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:27.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.499+0000 7f32a6b56640 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ee4f0 tx=0x7f329c0b7f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9846b7d5b4860d9a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f328bfff640 1 -- 172.21.15.96:0/3474809050 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0bdb20 con 0x55cdeaefb3b0 2024-03-18T15:43:27.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f328bfff640 1 -- 172.21.15.96:0/3474809050 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c0bd420 con 0x55cdeaefb3b0 2024-03-18T15:43:27.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f328bfff640 1 -- 172.21.15.96:0/3474809050 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0c1170 con 0x55cdeaefb3b0 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 -- 172.21.15.96:0/3474809050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 msgr2=0x55cdeaefedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ee4f0 tx=0x7f329c0b7f90 comp rx=0 tx=0).stop 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 -- 172.21.15.96:0/3474809050 shutdown_connections 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaefedf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/3474809050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaefe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.500+0000 7f32ab1a4740 1 -- 172.21.15.96:0/3474809050 >> 172.21.15.96:0/3474809050 conn(0x55cdeaef0dc0 msgr2=0x55cdeaef48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- 172.21.15.96:0/3474809050 shutdown_connections 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- 172.21.15.96:0/3474809050 wait complete. 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- start start 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 Processor -- start 2024-03-18T15:43:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaf04950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01aa200 con 0x55cdeaefb3b0 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01a9b30 con 0x55cdeaef6240 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01a6ce0 con 0x55cdeaef5850 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32ab1a4740 10 client.0 _ll_register_callbacks cb 0x7f329800f7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a6b56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a6b56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40204/0 (socket says 172.21.15.96:40204) 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a6b56640 1 -- 172.21.15.96:0/2653031582 learned_addr learned my addr 172.21.15.96:0/2653031582 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a5b54640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a6355640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaf04950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.501+0000 7f32a6b56640 1 -- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 msgr2=0x55cdeaf04950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32a6b56640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaf04950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32a6b56640 1 -- 172.21.15.96:0/2653031582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 msgr2=0x55cdeaf04e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32a6b56640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf04e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32a6b56640 1 -- 172.21.15.96:0/2653031582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdeaf06440 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32a6b56640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f329c0036a0 tx=0x7f329c02c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0908d35a20e87fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0da020 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f329c0c1170 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c0bd420 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f329c0c1370 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 10 client.16506 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdeaf72090 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c0bd420 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f329c09e070 con 0x55cdeaefb3b0 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 10 client.16506 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 10 client.16506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 20 client.16506 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf729d0 0x55cdeaf74df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 4 client.16506 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.502+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdeae61d20 con 0x55cdeaf729d0 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32a6355640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf729d0 0x55cdeaf74df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f3289ffb640 10 client.16506 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32a6355640 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf729d0 0x55cdeaf74df0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100e1ff1de6e6413 server_cookie=4bb9ad5a6762bfef in_seq=0 out_seq=0 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f3289ffb640 1 -- 172.21.15.96:0/2653031582 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdeae61d20 con 0x55cdeaf729d0 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f3289ffb640 10 client.16506 handle_command_reply: tid=0 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 client.16506 shutdown 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 msgr2=0x55cdeaf05e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f329c0036a0 tx=0x7f329c02c880 comp rx=0 tx=0).stop 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 shutdown_connections 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55cdeaf729d0 0x55cdeaf74df0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f3289ffb640 0 client.16506 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cdeaefb3b0 0x55cdeaf05e50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cdeaef6240 0x55cdeaf04e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 --2- 172.21.15.96:0/2653031582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cdeaef5850 0x55cdeaf04950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 >> 172.21.15.96:0/2653031582 conn(0x55cdeaef0dc0 msgr2=0x55cdeaeef750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.503+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 shutdown_connections 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32ab1a4740 1 -- 172.21.15.96:0/2653031582 wait complete. 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32ab1a4740 20 client.16506 trim_cache size 0 max 16384 2024-03-18T15:43:27.510 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 msgr2=0x7f328c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 0x7f328c0a4cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55cdeaef6640 tx=0x7f3298005f90 comp rx=0 tx=0).stop 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 msgr2=0x7f32a019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 0x7f32a019f070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f329c02c170 tx=0x7f329c02c640 comp rx=0 tx=0).stop 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 shutdown_connections 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f328c0a2800 0x7f328c0a4cc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a0102290 0x7f32a019f070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32a01018d0 0x7f32a0198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 --2- 172.21.15.96:0/3720974544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f32a01006e0 0x7f32a0197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:27.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 >> 172.21.15.96:0/3720974544 conn(0x7f32a00fbc70 msgr2=0x7f32a0068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:27.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 shutdown_connections 2024-03-18T15:43:27.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:27.504+0000 7f32a85b9640 1 -- 172.21.15.96:0/3720974544 wait complete. 2024-03-18T15:43:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:27 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2522749034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:43:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:27 smithi096 ceph-mon[31930]: pgmap v453: 97 pgs: 97 active+clean; 386 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-03-18T15:43:28.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:27 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2522749034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:43:28.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:27 smithi137 ceph-mon[35868]: pgmap v453: 97 pgs: 97 active+clean; 386 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-03-18T15:43:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:27 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2522749034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:43:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:27 smithi193 ceph-mon[36702]: pgmap v453: 97 pgs: 97 active+clean; 386 MiB data, 783 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 249 op/s 2024-03-18T15:43:28.518 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:43:28.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:43:28.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:43:28.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.625+0000 7ff78a9b2640 1 -- 172.21.15.96:0/2767037377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841006e0 msgr2=0x7ff784100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.625+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/2767037377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841006e0 0x7ff784100b00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 -- 172.21.15.96:0/2767037377 shutdown_connections 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/2767037377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 0x7ff784108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/2767037377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841018d0 0x7ff784101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/2767037377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841006e0 0x7ff784100b00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 -- 172.21.15.96:0/2767037377 >> 172.21.15.96:0/2767037377 conn(0x7ff7840fbc70 msgr2=0x7ff7840fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 -- 172.21.15.96:0/2767037377 shutdown_connections 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 -- 172.21.15.96:0/2767037377 wait complete. 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 Processor -- start 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.626+0000 7ff78a9b2640 1 -- start start 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 0x7ff78419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff78410c360 con 0x7ff7841018d0 2024-03-18T15:43:28.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff78410c1e0 con 0x7ff7841006e0 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff78a9b2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff78410c4e0 con 0x7ff784102290 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff783fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff788f4f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 0x7ff78419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40228/0 (socket says 172.21.15.96:40228) 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 -- 172.21.15.96:0/992408199 learned_addr learned my addr 172.21.15.96:0/992408199 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff783fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42204/0 (socket says 172.21.15.96:42204) 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 -- 172.21.15.96:0/992408199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 msgr2=0x7ff78419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 0x7ff78419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 -- 172.21.15.96:0/992408199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 msgr2=0x7ff784197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 -- 172.21.15.96:0/992408199 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78419f730 con 0x7ff7841018d0 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff783fff640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:28.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7837fe640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff77800e9f0 tx=0x7ff77800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.627+0000 7ff7817fa640 1 -- 172.21.15.96:0/992408199 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77800cd80 con 0x7ff7841018d0 2024-03-18T15:43:28.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.628+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78419fa20 con 0x7ff7841018d0 2024-03-18T15:43:28.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.628+0000 7ff7817fa640 1 -- 172.21.15.96:0/992408199 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff778010910 con 0x7ff7841018d0 2024-03-18T15:43:28.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.628+0000 7ff7817fa640 1 -- 172.21.15.96:0/992408199 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7780188f0 con 0x7ff7841018d0 2024-03-18T15:43:28.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.628+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78419ffc0 con 0x7ff7841018d0 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.629+0000 7ff7817fa640 1 -- 172.21.15.96:0/992408199 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff778010ac0 con 0x7ff7841018d0 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 Processor -- start 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff7817fa640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 0x7ff76c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 -- start start 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff788f4f640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 0x7ff76c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff7817fa640 1 -- 172.21.15.96:0/992408199 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff7780cee30 con 0x7ff7841018d0 2024-03-18T15:43:28.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55caf504e0d0 con 0x55caf50e9990 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55caf4ff6c70 con 0x55caf50ea380 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff78d59e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55caf504a170 con 0x55caf50ef4f0 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40238/0 (socket says 172.21.15.96:40238) 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff783fff640 1 -- 172.21.15.96:0/3992047533 learned_addr learned my addr 172.21.15.96:0/3992047533 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff788f4f640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.630+0000 7ff7837fe640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 -- 172.21.15.96:0/3992047533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 msgr2=0x55caf50f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 -- 172.21.15.96:0/3992047533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 msgr2=0x55caf50eee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50eee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff788f4f640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 0x7ff76c0a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005fd0 tx=0x7ff77c011bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 -- 172.21.15.96:0/3992047533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf4e2a7e0 con 0x55caf50e9990 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff7837fe640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff783fff640 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2988ca5001121907 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff76a7fc640 1 -- 172.21.15.96:0/3992047533 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774005d50 con 0x55caf50e9990 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff76a7fc640 1 -- 172.21.15.96:0/3992047533 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff774005520 con 0x55caf50e9990 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.631+0000 7ff76a7fc640 1 -- 172.21.15.96:0/3992047533 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774004c20 con 0x55caf50e9990 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- 172.21.15.96:0/3992047533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 msgr2=0x55caf50e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff774009900 tx=0x7ff77402cd30 comp rx=0 tx=0).stop 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- 172.21.15.96:0/3992047533 shutdown_connections 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50eee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- 172.21.15.96:0/3992047533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50e9db0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- 172.21.15.96:0/3992047533 >> 172.21.15.96:0/3992047533 conn(0x55caf50b1c50 msgr2=0x55caf50e5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- 172.21.15.96:0/3992047533 shutdown_connections 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- 172.21.15.96:0/3992047533 wait complete. 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- start start 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 Processor -- start 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ea380 0x55caf50f4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ef4f0 0x55caf50fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55caf50f2440 con 0x55caf50e9990 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55caf500a0d0 con 0x55caf50ef4f0 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.632+0000 7ff78d59e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55caf50f25c0 con 0x55caf50ea380 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff78d59e740 10 client.0 _ll_register_callbacks cb 0x55caf4fd3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40244/0 (socket says 172.21.15.96:40244) 2024-03-18T15:43:28.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 -- 172.21.15.96:0/2268328375 learned_addr learned my addr 172.21.15.96:0/2268328375 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff7837fe640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ea380 0x55caf50f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff788f4f640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ef4f0 0x55caf50fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 -- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ea380 msgr2=0x55caf50f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ea380 0x55caf50f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 -- 172.21.15.96:0/2268328375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ef4f0 msgr2=0x55caf50fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ef4f0 0x55caf50fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 -- 172.21.15.96:0/2268328375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf4e8af70 con 0x55caf50e9990 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff783fff640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff77402c7b0 tx=0x7ff774004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117ccf43ec24093f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff7637fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774005d50 con 0x55caf50e9990 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff774003a00 con 0x55caf50e9990 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff774005520 con 0x55caf50e9990 2024-03-18T15:43:28.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff774003c00 con 0x55caf50e9990 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff78d59e740 10 client.16518 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.633+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55caf4d22170 con 0x55caf50e9990 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff774005520 con 0x55caf50e9990 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff774034440 con 0x55caf50e9990 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 10 client.16518 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 20 client.16518 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf50ff520 0x55caf5101910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 4 client.16518 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf505b610 con 0x55caf50ff520 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff7837fe640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf50ff520 0x55caf5101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff7837fe640 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf50ff520 0x55caf5101910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d0d19c679ff76f server_cookie=47358523789af667 in_seq=0 out_seq=0 2024-03-18T15:43:28.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.634+0000 7ff7637fe640 10 client.16518 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.635+0000 7ff7637fe640 1 -- 172.21.15.96:0/2268328375 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55caf505b610 con 0x55caf50ff520 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.635+0000 7ff7637fe640 10 client.16518 handle_command_reply: tid=0 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.635+0000 7ff78d59e740 1 client.16518 shutdown 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 msgr2=0x55caf50f3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff77402c7b0 tx=0x7ff774004610 comp rx=0 tx=0).stop 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 shutdown_connections 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ef4f0 0x55caf50fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ea380 0x55caf50f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf50ff520 0x55caf5101910 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 --2- 172.21.15.96:0/2268328375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf50f3e40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 >> 172.21.15.96:0/2268328375 conn(0x55caf50b1c50 msgr2=0x55caf50e44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff7637fe640 0 client.16518 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 shutdown_connections 2024-03-18T15:43:28.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 1 -- 172.21.15.96:0/2268328375 wait complete. 2024-03-18T15:43:28.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.636+0000 7ff78d59e740 20 client.16518 trim_cache size 0 max 16384 2024-03-18T15:43:28.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 Processor -- start 2024-03-18T15:43:28.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 -- start start 2024-03-18T15:43:28.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50fb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841a8cd0 con 0x55caf50e9990 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841aa8a0 con 0x55caf50ea380 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff78d59e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841a93a0 con 0x55caf50ef4f0 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40258/0 (socket says 172.21.15.96:40258) 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 -- 172.21.15.96:0/696541937 learned_addr learned my addr 172.21.15.96:0/696541937 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff7837fe640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff788f4f640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 -- 172.21.15.96:0/696541937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 msgr2=0x55caf50e8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50e8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 -- 172.21.15.96:0/696541937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 msgr2=0x55caf50fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 -- 172.21.15.96:0/696541937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf50e58a0 con 0x55caf50e9990 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff788f4f640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff783fff640 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff77402c8e0 tx=0x7ff774009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f717f980b19fb6e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.654+0000 7ff769ffb640 1 -- 172.21.15.96:0/696541937 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774005d50 con 0x55caf50e9990 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff769ffb640 1 -- 172.21.15.96:0/696541937 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77404d070 con 0x55caf50e9990 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff769ffb640 1 -- 172.21.15.96:0/696541937 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774034440 con 0x55caf50e9990 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 -- 172.21.15.96:0/696541937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 msgr2=0x55caf5107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff77402c8e0 tx=0x7ff774009620 comp rx=0 tx=0).stop 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 -- 172.21.15.96:0/696541937 shutdown_connections 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50ef4f0 0x55caf50e8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50fb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 --2- 172.21.15.96:0/696541937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50e9990 0x55caf5107120 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 -- 172.21.15.96:0/696541937 >> 172.21.15.96:0/696541937 conn(0x55caf50b1c50 msgr2=0x55caf50e8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 -- 172.21.15.96:0/696541937 shutdown_connections 2024-03-18T15:43:28.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.655+0000 7ff78d59e740 1 -- 172.21.15.96:0/696541937 wait complete. 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 -- start start 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 Processor -- start 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50e9990 0x55caf50f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841aa1d0 con 0x55caf50ef4f0 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841a9b00 con 0x55caf50ea380 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7841a6cb0 con 0x55caf50e9990 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff78d59e740 10 client.0 _ll_register_callbacks cb 0x7ff7780973f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff783fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50e9990 0x55caf50f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40264/0 (socket says 172.21.15.96:40264) 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 -- 172.21.15.96:0/1946411206 learned_addr learned my addr 172.21.15.96:0/1946411206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff7837fe640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 -- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50e9990 msgr2=0x55caf50f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50e9990 0x55caf50f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 -- 172.21.15.96:0/1946411206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 msgr2=0x55caf50f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.656+0000 7ff788f4f640 1 -- 172.21.15.96:0/1946411206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf50fa6d0 con 0x55caf50ef4f0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff788f4f640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00ff40 tx=0x7ff77c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f024728e12e3de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c01a040 con 0x55caf50ef4f0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff77c01f7c0 con 0x55caf50ef4f0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff77404d070 con 0x55caf50ef4f0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff77c01f9c0 con 0x55caf50ef4f0 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 10 client.16524 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:28.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55caf5165910 con 0x55caf50ef4f0 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77404d070 con 0x55caf50ef4f0 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff77c01d030 con 0x55caf50ef4f0 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 10 client.16524 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 20 client.16524 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf5166390 0x55caf51687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 4 client.16524 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.657+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55caf5055960 con 0x55caf5166390 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.658+0000 7ff783fff640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf5166390 0x55caf51687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.658+0000 7ff783fff640 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf5166390 0x55caf51687b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ff59c8a4c39376 server_cookie=456fe150d2a4a097 in_seq=0 out_seq=0 2024-03-18T15:43:28.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.658+0000 7ff7637fe640 10 client.16524 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff7637fe640 1 -- 172.21.15.96:0/1946411206 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55caf5055960 con 0x55caf5166390 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff7637fe640 10 client.16524 handle_command_reply: tid=0 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 client.16524 shutdown 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 msgr2=0x55caf50fa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00ff40 tx=0x7ff77c004ab0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 shutdown_connections 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55caf5166390 0x55caf51687b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff7637fe640 0 client.16524 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caf50ef4f0 0x55caf50fa0e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55caf50ea380 0x55caf50f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 --2- 172.21.15.96:0/1946411206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55caf50e9990 0x55caf50f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 >> 172.21.15.96:0/1946411206 conn(0x55caf50b1c50 msgr2=0x55caf50e8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 shutdown_connections 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.659+0000 7ff78d59e740 1 -- 172.21.15.96:0/1946411206 wait complete. 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78d59e740 20 client.16524 trim_cache size 0 max 16384 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "201eed03-ac93-4b1d-816a-172ace6d3466", 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:43:28.667 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 msgr2=0x7ff76c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 0x7ff76c0a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005fd0 tx=0x7ff77c011bc0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 msgr2=0x7ff7841980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff77800e9f0 tx=0x7ff77800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 shutdown_connections 2024-03-18T15:43:28.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff76c0a2800 0x7ff76c0a4cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff784102290 0x7ff78419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7841018d0 0x7ff7841980a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 --2- 172.21.15.96:0/992408199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7841006e0 0x7ff784197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 >> 172.21.15.96:0/992408199 conn(0x7ff7840fbc70 msgr2=0x7ff7840fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 shutdown_connections 2024-03-18T15:43:28.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.660+0000 7ff78a9b2640 1 -- 172.21.15.96:0/992408199 wait complete. 2024-03-18T15:43:28.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:43:28.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 -- 172.21.15.96:0/4146442014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac8102290 msgr2=0x7f4ac8108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 --2- 172.21.15.96:0/4146442014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac8102290 0x7f4ac8108b20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009ca0 tx=0x7f4ac002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 -- 172.21.15.96:0/4146442014 shutdown_connections 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 --2- 172.21.15.96:0/4146442014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac8102290 0x7f4ac8108b20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 --2- 172.21.15.96:0/4146442014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac81018d0 0x7f4ac8101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 --2- 172.21.15.96:0/4146442014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81006e0 0x7f4ac8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 -- 172.21.15.96:0/4146442014 >> 172.21.15.96:0/4146442014 conn(0x7f4ac80fbc70 msgr2=0x7f4ac80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 -- 172.21.15.96:0/4146442014 shutdown_connections 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.779+0000 7f4acce3f640 1 -- 172.21.15.96:0/4146442014 wait complete. 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 Processor -- start 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 -- start start 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81018d0 0x7f4ac8196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8102290 0x7f4ac819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac810c360 con 0x7f4ac81006e0 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac810c1e0 con 0x7f4ac8102290 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4acce3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac810c4e0 con 0x7f4ac81018d0 2024-03-18T15:43:28.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40278/0 (socket says 172.21.15.96:40278) 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 -- 172.21.15.96:0/1995165191 learned_addr learned my addr 172.21.15.96:0/1995165191 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81018d0 0x7f4ac8196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8102290 0x7f4ac819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 -- 172.21.15.96:0/1995165191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81018d0 msgr2=0x7f4ac8196f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81018d0 0x7f4ac8196f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 -- 172.21.15.96:0/1995165191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8102290 msgr2=0x7f4ac819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8102290 0x7f4ac819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 -- 172.21.15.96:0/1995165191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac819e5f0 con 0x7f4ac81006e0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.780+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9c0 tx=0x7f4ab800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.781+0000 7f4ab77fe640 1 -- 172.21.15.96:0/1995165191 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800cd80 con 0x7f4ac81006e0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.781+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac819e8e0 con 0x7f4ac81006e0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.781+0000 7f4ab77fe640 1 -- 172.21.15.96:0/1995165191 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab80035f0 con 0x7f4ac81006e0 2024-03-18T15:43:28.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.781+0000 7f4ab77fe640 1 -- 172.21.15.96:0/1995165191 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8010690 con 0x7f4ac81006e0 2024-03-18T15:43:28.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.781+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac819ee30 con 0x7f4ac81006e0 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ab77fe640 1 -- 172.21.15.96:0/1995165191 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4ab80037a0 con 0x7f4ac81006e0 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 Processor -- start 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 -- start start 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f3f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f3fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56419f35b500 con 0x56419f3fbcd0 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56419f303be0 con 0x56419f3f6b60 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4acfa20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56419f3575a0 con 0x56419f3f6170 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ab77fe640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f3fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ac5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40288/0 (socket says 172.21.15.96:40288) 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.783+0000 7f4ac5d9b640 1 -- 172.21.15.96:0/1206682838 learned_addr learned my addr 172.21.15.96:0/1206682838 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f3f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 -- 172.21.15.96:0/1206682838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 msgr2=0x56419f3f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f3f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 -- 172.21.15.96:0/1206682838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 msgr2=0x56419f3fe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ab77fe640 1 -- 172.21.15.96:0/1995165191 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4ab80cedf0 con 0x7f4ac81006e0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 -- 172.21.15.96:0/1206682838 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419f3174b0 con 0x56419f3f6b60 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005e30 tx=0x7f4ac0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ac659c640 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f3fb660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80993d0 tx=0x7f4ab8097400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f09ca975b9b962 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ab4ff9640 1 -- 172.21.15.96:0/1206682838 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab809b760 con 0x56419f3f6b60 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ab4ff9640 1 -- 172.21.15.96:0/1206682838 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab809f8b0 con 0x56419f3f6b60 2024-03-18T15:43:28.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.784+0000 7f4ab4ff9640 1 -- 172.21.15.96:0/1206682838 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8097dd0 con 0x56419f3f6b60 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- 172.21.15.96:0/1206682838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 msgr2=0x56419f3fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f3fb660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80993d0 tx=0x7f4ab8097400 comp rx=0 tx=0).stop 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- 172.21.15.96:0/1206682838 shutdown_connections 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f3fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f3fb660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 --2- 172.21.15.96:0/1206682838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f3f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- 172.21.15.96:0/1206682838 >> 172.21.15.96:0/1206682838 conn(0x56419f3f16e0 msgr2=0x56419f3f1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- 172.21.15.96:0/1206682838 shutdown_connections 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- 172.21.15.96:0/1206682838 wait complete. 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 -- start start 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.785+0000 7f4acfa20740 1 Processor -- start 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 0x56419f400710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f400c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56419f3fec20 con 0x56419f3fbcd0 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56419f317160 con 0x56419f3f6170 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56419f3feda0 con 0x56419f3f6b60 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 10 client.0 _ll_register_callbacks cb 0x56419f2fb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 0x56419f400710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40304/0 (socket says 172.21.15.96:40304) 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac6d9d640 1 -- 172.21.15.96:0/2181269094 learned_addr learned my addr 172.21.15.96:0/2181269094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f400c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 -- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 msgr2=0x56419f400c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f400c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 -- 172.21.15.96:0/2181269094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 msgr2=0x56419f407bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 -- 172.21.15.96:0/2181269094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419f1397d0 con 0x56419f3f6170 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 0x56419f400710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56419f3f6f60 tx=0x7f4ab8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4671f436ad260d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4aadffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80a3f00 con 0x56419f3f6170 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4ab8099400 con 0x56419f3f6170 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.786+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab809f8b0 con 0x56419f3f6170 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ab8099640 con 0x56419f3f6170 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 10 client.26330 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:28.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56419f06f3f0 con 0x56419f3f6170 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab809f8b0 con 0x56419f3f6170 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4ab80a3a70 con 0x56419f3f6170 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 10 client.26330 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 20 client.26330 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f40bd10 0x56419f40e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 4 client.26330 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419f368a40 con 0x56419f40bd10 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.787+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f40bd10 0x56419f40e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.788+0000 7f4aadffb640 10 client.26330 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.788+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f40bd10 0x56419f40e100 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659b8bb877777d4d server_cookie=3daa79a30f869651 in_seq=0 out_seq=0 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.788+0000 7f4aadffb640 1 -- 172.21.15.96:0/2181269094 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56419f368a40 con 0x56419f40bd10 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.788+0000 7f4aadffb640 10 client.26330 handle_command_reply: tid=0 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 client.26330 shutdown 2024-03-18T15:43:28.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 msgr2=0x56419f400710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 0x56419f400710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x56419f3f6f60 tx=0x7f4ab8003f10 comp rx=0 tx=0).stop 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 shutdown_connections 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f40bd10 0x56419f40e100 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f407bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f400c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2181269094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6170 0x56419f400710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4aadffb640 0 client.26330 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 >> 172.21.15.96:0/2181269094 conn(0x56419f3f16e0 msgr2=0x56419f3f0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 shutdown_connections 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 1 -- 172.21.15.96:0/2181269094 wait complete. 2024-03-18T15:43:28.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.789+0000 7f4acfa20740 20 client.26330 trim_cache size 0 max 16384 2024-03-18T15:43:28.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.806+0000 7f4acfa20740 1 Processor -- start 2024-03-18T15:43:28.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 -- start start 2024-03-18T15:43:28.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f407b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3fbcd0 0x56419f3f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a7c20 con 0x56419f3f6170 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a97f0 con 0x56419f3fbcd0 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4acfa20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a82f0 con 0x56419f3f6b60 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40308/0 (socket says 172.21.15.96:40308) 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 -- 172.21.15.96:0/2789843803 learned_addr learned my addr 172.21.15.96:0/2789843803 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f407b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3fbcd0 0x56419f3f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 -- 172.21.15.96:0/2789843803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 msgr2=0x56419f407b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f407b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 -- 172.21.15.96:0/2789843803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3fbcd0 msgr2=0x56419f3f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3fbcd0 0x56419f3f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 -- 172.21.15.96:0/2789843803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419f02f0f0 con 0x56419f3f6170 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4ac659c640 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8097bc0 tx=0x7f4ab809b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a23805ed8f8c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4aaffff640 1 -- 172.21.15.96:0/2789843803 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80a3f00 con 0x56419f3f6170 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4aaffff640 1 -- 172.21.15.96:0/2789843803 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab809f8b0 con 0x56419f3f6170 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.807+0000 7f4aaffff640 1 -- 172.21.15.96:0/2789843803 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab809b9c0 con 0x56419f3f6170 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- 172.21.15.96:0/2789843803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 msgr2=0x56419f400a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8097bc0 tx=0x7f4ab809b5b0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- 172.21.15.96:0/2789843803 shutdown_connections 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3fbcd0 0x56419f3f0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6b60 0x56419f407b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 --2- 172.21.15.96:0/2789843803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3f6170 0x56419f400a30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- 172.21.15.96:0/2789843803 >> 172.21.15.96:0/2789843803 conn(0x56419f3f16e0 msgr2=0x56419f400320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- 172.21.15.96:0/2789843803 shutdown_connections 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- 172.21.15.96:0/2789843803 wait complete. 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 -- start start 2024-03-18T15:43:28.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.808+0000 7f4acfa20740 1 Processor -- start 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f404d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f405250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a9120 con 0x56419f3fbcd0 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a8a50 con 0x56419f3f6b60 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81a5c00 con 0x56419f3f6170 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 10 client.0 _ll_register_callbacks cb 0x7f4abc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40324/0 (socket says 172.21.15.96:40324) 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 -- 172.21.15.96:0/704317227 learned_addr learned my addr 172.21.15.96:0/704317227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac659c640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f404d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac5d9b640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f405250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 -- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 msgr2=0x56419f404d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f404d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 -- 172.21.15.96:0/704317227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 msgr2=0x56419f405250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f405250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 -- 172.21.15.96:0/704317227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419f3229d0 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4ac6d9d640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009ca0 tx=0x7f4ac00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ddad3059b7abc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4aadffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0039040 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4ac0004690 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.809+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab809f8b0 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ac0004b40 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 10 client.16542 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56419f407a00 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab809f8b0 con 0x56419f3fbcd0 2024-03-18T15:43:28.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f4ac0005890 con 0x56419f3fbcd0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 10 client.16542 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 20 client.16542 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f4767b0 0x56419f478bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 4 client.16542 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56419f362d90 con 0x56419f4767b0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4ac659c640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f4767b0 0x56419f478bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4ac659c640 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f4767b0 0x56419f478bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a3d2d5163e13138 server_cookie=e26b7aa4f2d75b6f in_seq=0 out_seq=0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.810+0000 7f4aadffb640 10 client.16542 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4aadffb640 1 -- 172.21.15.96:0/704317227 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x56419f362d90 con 0x56419f4767b0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4aadffb640 10 client.16542 handle_command_reply: tid=0 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 client.16542 shutdown 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 msgr2=0x56419f406260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009ca0 tx=0x7f4ac00095c0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 shutdown_connections 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56419f4767b0 0x56419f478bd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56419f3fbcd0 0x56419f406260 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56419f3f6b60 0x56419f405250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4aadffb640 0 client.16542 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 --2- 172.21.15.96:0/704317227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56419f3f6170 0x56419f404d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 >> 172.21.15.96:0/704317227 conn(0x56419f3f16e0 msgr2=0x56419f3f4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 shutdown_connections 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 1 -- 172.21.15.96:0/704317227 wait complete. 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acfa20740 20 client.16542 trim_cache size 0 max 16384 2024-03-18T15:43:28.820 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "scrub active (0 inodes in the stack)", 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "201eed03-ac93-4b1d-816a-172ace6d3466": { 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "201eed03-ac93-4b1d-816a-172ace6d3466", 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:43:28.821 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:43:28.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 msgr2=0x7f4ab00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005e30 tx=0x7f4ac0032590 comp rx=0 tx=0).stop 2024-03-18T15:43:28.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 msgr2=0x7f4ac81969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9c0 tx=0x7f4ab800ee80 comp rx=0 tx=0).stop 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 shutdown_connections 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4ab00a2800 0x7f4ab00a4cc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4ac8102290 0x7f4ac819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac81018d0 0x7f4ac8196f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 --2- 172.21.15.96:0/1995165191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ac81006e0 0x7f4ac81969d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.811+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 >> 172.21.15.96:0/1995165191 conn(0x7f4ac80fbc70 msgr2=0x7f4ac8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.812+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 shutdown_connections 2024-03-18T15:43:28.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:28.812+0000 7f4acce3f640 1 -- 172.21.15.96:0/1995165191 wait complete. 2024-03-18T15:43:28.823 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:28 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:28.824 INFO:tasks.cephfs.filesystem:scrub status for tag:201eed03-ac93-4b1d-816a-172ace6d3466 - {'path': '/', 'tag': '201eed03-ac93-4b1d-816a-172ace6d3466', 'options': 'recursive,force'} 2024-03-18T15:43:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:28 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:28 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:30.070 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.070 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:29 smithi193 ceph-mon[36702]: pgmap v454: 97 pgs: 97 active+clean; 412 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-03-18T15:43:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:29 smithi137 ceph-mon[35868]: pgmap v454: 97 pgs: 97 active+clean; 412 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-03-18T15:43:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:43:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:30.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:29 smithi096 ceph-mon[31930]: pgmap v454: 97 pgs: 97 active+clean; 412 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 420 op/s 2024-03-18T15:43:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:30 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:43:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:30 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:30 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:43:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:30 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:43:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:30 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:30 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:43:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:30 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:43:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:30 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:43:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:30 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:43:31.328 INFO:tasks.workunit.client.0.smithi096.stdout: 81 1044347 192 771609 196 588674 610 2024-03-18T15:43:31.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.581+0000 7fa7ac35b640 1 -- 172.21.15.96:0/99400861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a4100700 msgr2=0x7fa7a4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.581+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/99400861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a4100700 0x7fa7a4100b20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009900 tx=0x7fa7a002bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- 172.21.15.96:0/99400861 shutdown_connections 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/99400861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41022b0 0x7fa7a4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/99400861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a41018f0 0x7fa7a4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/99400861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a4100700 0x7fa7a4100b20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- 172.21.15.96:0/99400861 >> 172.21.15.96:0/99400861 conn(0x7fa7a40fbc70 msgr2=0x7fa7a40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- 172.21.15.96:0/99400861 shutdown_connections 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- 172.21.15.96:0/99400861 wait complete. 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 Processor -- start 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- start start 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 0x7fa7a4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41018f0 0x7fa7a4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410c350 con 0x7fa7a41022b0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410c1d0 con 0x7fa7a4100700 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.582+0000 7fa7ac35b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a410c4d0 con 0x7fa7a41018f0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 0x7fa7a4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40340/0 (socket says 172.21.15.96:40340) 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 0x7fa7a4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42314/0 (socket says 172.21.15.96:42314) 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41018f0 0x7fa7a4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa0f7640 1 -- 172.21.15.96:0/2198796066 learned_addr learned my addr 172.21.15.96:0/2198796066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/2198796066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41018f0 msgr2=0x7fa7a4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41018f0 0x7fa7a4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/2198796066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 msgr2=0x7fa7a4197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 0x7fa7a4197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/2198796066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a419f6e0 con 0x7fa7a41022b0 2024-03-18T15:43:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00da10 tx=0x7fa79c00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa79b7fe640 1 -- 172.21.15.96:0/2198796066 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00bd70 con 0x7fa7a41022b0 2024-03-18T15:43:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa79b7fe640 1 -- 172.21.15.96:0/2198796066 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa79c003e50 con 0x7fa7a41022b0 2024-03-18T15:43:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa79b7fe640 1 -- 172.21.15.96:0/2198796066 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c0106a0 con 0x7fa7a41022b0 2024-03-18T15:43:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a419f9d0 con 0x7fa7a41022b0 2024-03-18T15:43:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.583+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a419ff20 con 0x7fa7a41022b0 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.585+0000 7fa79b7fe640 1 -- 172.21.15.96:0/2198796066 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fa79c003970 con 0x7fa7a41022b0 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 Processor -- start 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 -- start start 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa79b7fe640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 0x7fa7900a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859beb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859bee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d859b4ba40 con 0x55d859be6d30 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d859af4920 con 0x55d859be6340 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7af024740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d859b47ae0 con 0x55d859bebea0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.586+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 0x7fa7900a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa79b7fe640 1 -- 172.21.15.96:0/2198796066 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa79c0cedf0 con 0x7fa7a41022b0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859beb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859bee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42320/0 (socket says 172.21.15.96:42320) 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 -- 172.21.15.96:0/3366154405 learned_addr learned my addr 172.21.15.96:0/3366154405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859beb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40342/0 (socket says 172.21.15.96:40342) 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 -- 172.21.15.96:0/3366154405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 msgr2=0x55d859bee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859bee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 -- 172.21.15.96:0/3366154405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 msgr2=0x55d859beb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859beb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 -- 172.21.15.96:0/3366154405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d859b0c460 con 0x55d859be6340 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 0x7fa7900a4de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4198fd0 tx=0x7fa794009b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002cfd0 tx=0x7fa7a0009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ec5d7a8828ffb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa798bfe640 1 -- 172.21.15.96:0/3366154405 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0042070 con 0x55d859be6340 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa798bfe640 1 -- 172.21.15.96:0/3366154405 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a002cc40 con 0x55d859be6340 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.587+0000 7fa798bfe640 1 -- 172.21.15.96:0/3366154405 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0005150 con 0x55d859be6340 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- 172.21.15.96:0/3366154405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 msgr2=0x55d859be6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002cfd0 tx=0x7fa7a0009610 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- 172.21.15.96:0/3366154405 shutdown_connections 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859bee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859beb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 --2- 172.21.15.96:0/3366154405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be6760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- 172.21.15.96:0/3366154405 >> 172.21.15.96:0/3366154405 conn(0x55d859be18b0 msgr2=0x55d859be1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- 172.21.15.96:0/3366154405 shutdown_connections 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- 172.21.15.96:0/3366154405 wait complete. 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.588+0000 7fa7af024740 1 -- start start 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 Processor -- start 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859bf0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6d30 0x55d859bf0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d859beedf0 con 0x55d859bebea0 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d859b07d70 con 0x55d859be6340 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d859beef70 con 0x55d859be6d30 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 10 client.0 _ll_register_callbacks cb 0x55d85985fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859bf0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40348/0 (socket says 172.21.15.96:40348) 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6d30 0x55d859bf0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/937584550 learned_addr learned my addr 172.21.15.96:0/937584550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6d30 msgr2=0x55d859bf0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6d30 0x55d859bf0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/937584550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 msgr2=0x55d859bf0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859bf0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/937584550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d859860210 con 0x55d859bebea0 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0cc910 tx=0x7fa79c09b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34f17cbce0e10ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c014070 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa79c0042e0 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.589+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7a002cc40 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa79c01aba0 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 10 client.16554 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d85992c620 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7a002cc40 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fa79c024060 con 0x55d859bebea0 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 10 client.16554 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 20 client.16554 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859bfc580 0x55d859bfe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 4 client.16554 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d859b58f80 con 0x55d859bfc580 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.590+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859bfc580 0x55d859bfe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.591+0000 7fa78dffb640 10 client.16554 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.591+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859bfc580 0x55d859bfe990 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6719f4dc95cf4f3d server_cookie=3e20b582b25743e5 in_seq=0 out_seq=0 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa78dffb640 1 -- 172.21.15.96:0/937584550 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d859b58f80 con 0x55d859bfc580 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa78dffb640 10 client.16554 handle_command_reply: tid=0 2024-03-18T15:43:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 client.16554 shutdown 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 msgr2=0x55d859bf7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0cc910 tx=0x7fa79c09b640 comp rx=0 tx=0).stop 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 shutdown_connections 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859bfc580 0x55d859bfe990 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf7b80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6d30 0x55d859bf0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa78dffb640 0 client.16554 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 --2- 172.21.15.96:0/937584550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859bf0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 >> 172.21.15.96:0/937584550 conn(0x55d859be18b0 msgr2=0x55d859be0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 shutdown_connections 2024-03-18T15:43:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 1 -- 172.21.15.96:0/937584550 wait complete. 2024-03-18T15:43:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.592+0000 7fa7af024740 20 client.16554 trim_cache size 0 max 16384 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 Processor -- start 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 -- start start 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859c03240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a8c80 con 0x55d859be6d30 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41aa850 con 0x55d859be6340 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7af024740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a9350 con 0x55d859bebea0 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40360/0 (socket says 172.21.15.96:40360) 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 -- 172.21.15.96:0/2501745549 learned_addr learned my addr 172.21.15.96:0/2501745549 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859c03240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 -- 172.21.15.96:0/2501745549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 msgr2=0x55d859c03240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859c03240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 -- 172.21.15.96:0/2501745549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 msgr2=0x55d859be5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 -- 172.21.15.96:0/2501745549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d859be6720 con 0x55d859be6d30 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.607+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa794009420 tx=0x7fa794021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf677a756796ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa78ffff640 1 -- 172.21.15.96:0/2501745549 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79402e070 con 0x55d859be6d30 2024-03-18T15:43:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa78ffff640 1 -- 172.21.15.96:0/2501745549 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7940097a0 con 0x55d859be6d30 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa78ffff640 1 -- 172.21.15.96:0/2501745549 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940296a0 con 0x55d859be6d30 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 -- 172.21.15.96:0/2501745549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 msgr2=0x55d859bf7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa794009420 tx=0x7fa794021000 comp rx=0 tx=0).stop 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 -- 172.21.15.96:0/2501745549 shutdown_connections 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859bebea0 0x55d859c03240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859be6d30 0x55d859bf7b40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 --2- 172.21.15.96:0/2501745549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6340 0x55d859be5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 -- 172.21.15.96:0/2501745549 >> 172.21.15.96:0/2501745549 conn(0x55d859be18b0 msgr2=0x55d859be4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 -- 172.21.15.96:0/2501745549 shutdown_connections 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.608+0000 7fa7af024740 1 -- 172.21.15.96:0/2501745549 wait complete. 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 -- start start 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 Processor -- start 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6340 0x55d859bf7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 0x55d859bf5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41aa180 con 0x55d859bebea0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a9ab0 con 0x55d859be6d30 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a41a6c60 con 0x55d859be6340 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7af024740 10 client.0 _ll_register_callbacks cb 0x7fa79c097a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa0f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6340 0x55d859bf7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40370/0 (socket says 172.21.15.96:40370) 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/538200021 learned_addr learned my addr 172.21.15.96:0/538200021 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 0x55d859bf5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6340 msgr2=0x55d859bf7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6340 0x55d859bf7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/538200021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 msgr2=0x55d859bf5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 0x55d859bf5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7aa8f8640 1 -- 172.21.15.96:0/538200021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d859bf67c0 con 0x55d859bebea0 2024-03-18T15:43:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.609+0000 7fa7a98f6640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 0x55d859bf5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7aa8f8640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa79c099a10 tx=0x7fa79c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1536985391bb1a0c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79c07f060 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7940097a0 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa79c0042e0 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa79c0b8030 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 10 client.16560 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d859c63540 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7940097a0 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7fa79c0a3a70 con 0x55d859bebea0 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 10 client.16560 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 20 client.16560 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859c63d30 0x55d859c66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 4 client.16560 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:43:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.610+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d859b532d0 con 0x55d859c63d30 2024-03-18T15:43:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.611+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859c63d30 0x55d859c66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.611+0000 7fa78dffb640 10 client.16560 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.611+0000 7fa7aa0f7640 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859c63d30 0x55d859c66150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4281ef82b7204e0a server_cookie=d3fc3ba95974656e in_seq=0 out_seq=0 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.623+0000 7fa78dffb640 1 -- 172.21.15.96:0/538200021 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d859b532d0 con 0x55d859c63d30 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.623+0000 7fa78dffb640 10 client.16560 handle_command_reply: tid=0 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 client.16560 shutdown 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 msgr2=0x55d859bf6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa79c099a10 tx=0x7fa79c00bf10 comp rx=0 tx=0).stop 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 shutdown_connections 2024-03-18T15:43:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d859c63d30 0x55d859c66150 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d859bebea0 0x55d859bf6280 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d859be6d30 0x55d859bf5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa78dffb640 0 client.16560 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 --2- 172.21.15.96:0/538200021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d859be6340 0x55d859bf7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 >> 172.21.15.96:0/538200021 conn(0x55d859be18b0 msgr2=0x55d859be4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 shutdown_connections 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 1 -- 172.21.15.96:0/538200021 wait complete. 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7af024740 20 client.16560 trim_cache size 0 max 16384 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:43:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:43:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:43:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:43:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 msgr2=0x7fa7900a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 0x7fa7900a4de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4198fd0 tx=0x7fa794009b60 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 msgr2=0x7fa7a419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.624+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00da10 tx=0x7fa79c00ded0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 shutdown_connections 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa7900a2920 0x7fa7900a4de0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7a41022b0 0x7fa7a419f070 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7a41018f0 0x7fa7a4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 --2- 172.21.15.96:0/2198796066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a4100700 0x7fa7a4197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 >> 172.21.15.96:0/2198796066 conn(0x7fa7a40fbc70 msgr2=0x7fa7a4068e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 shutdown_connections 2024-03-18T15:43:31.628 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:31.625+0000 7fa7ac35b640 1 -- 172.21.15.96:0/2198796066 wait complete. 2024-03-18T15:43:32.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:43:32.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:43:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:43:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:31 smithi137 ceph-mon[35868]: pgmap v455: 97 pgs: 97 active+clean; 419 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 423 op/s 2024-03-18T15:43:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:43:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:43:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:43:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:31 smithi193 ceph-mon[36702]: pgmap v455: 97 pgs: 97 active+clean; 419 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 423 op/s 2024-03-18T15:43:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:43:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:43:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:43:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:31 smithi096 ceph-mon[31930]: pgmap v455: 97 pgs: 97 active+clean; 419 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 423 op/s 2024-03-18T15:43:34.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:34 smithi137 ceph-mon[35868]: pgmap v456: 97 pgs: 97 active+clean; 421 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-03-18T15:43:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:34 smithi193 ceph-mon[36702]: pgmap v456: 97 pgs: 97 active+clean; 421 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-03-18T15:43:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:34 smithi096 ceph-mon[31930]: pgmap v456: 97 pgs: 97 active+clean; 421 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-03-18T15:43:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:36 smithi137 ceph-mon[35868]: pgmap v457: 97 pgs: 97 active+clean; 424 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 421 op/s 2024-03-18T15:43:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:36 smithi193 ceph-mon[36702]: pgmap v457: 97 pgs: 97 active+clean; 424 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 421 op/s 2024-03-18T15:43:36.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:36 smithi096 ceph-mon[31930]: pgmap v457: 97 pgs: 97 active+clean; 424 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 421 op/s 2024-03-18T15:43:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:38 smithi137 ceph-mon[35868]: pgmap v458: 97 pgs: 97 active+clean; 424 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-03-18T15:43:38.921 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:38 smithi193 ceph-mon[36702]: pgmap v458: 97 pgs: 97 active+clean; 424 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-03-18T15:43:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:38 smithi096 ceph-mon[31930]: pgmap v458: 97 pgs: 97 active+clean; 424 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 4.8 MiB/s wr, 361 op/s 2024-03-18T15:43:40.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:40 smithi137 ceph-mon[35868]: pgmap v459: 97 pgs: 97 active+clean; 428 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2024-03-18T15:43:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:40 smithi193 ceph-mon[36702]: pgmap v459: 97 pgs: 97 active+clean; 428 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2024-03-18T15:43:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:40 smithi096 ceph-mon[31930]: pgmap v459: 97 pgs: 97 active+clean; 428 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 280 op/s 2024-03-18T15:43:41.328 INFO:tasks.workunit.client.0.smithi096.stdout: 87 1496200 388 1113873 408 858008 1122 2024-03-18T15:43:42.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:42 smithi137 ceph-mon[35868]: pgmap v460: 97 pgs: 97 active+clean; 428 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-18T15:43:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:42 smithi193 ceph-mon[36702]: pgmap v460: 97 pgs: 97 active+clean; 428 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-18T15:43:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:42 smithi096 ceph-mon[31930]: pgmap v460: 97 pgs: 97 active+clean; 428 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-03-18T15:43:44.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 -- 172.21.15.96:0/2578042300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc1018f0 msgr2=0x7f45dc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 --2- 172.21.15.96:0/2578042300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc1018f0 0x7f45dc101d70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009900 tx=0x7f45d002bc30 comp rx=0 tx=0).stop 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 -- 172.21.15.96:0/2578042300 shutdown_connections 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 --2- 172.21.15.96:0/2578042300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 --2- 172.21.15.96:0/2578042300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc1018f0 0x7f45dc101d70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 --2- 172.21.15.96:0/2578042300 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc100700 0x7f45dc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 -- 172.21.15.96:0/2578042300 >> 172.21.15.96:0/2578042300 conn(0x7f45dc0fbc70 msgr2=0x7f45dc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 -- 172.21.15.96:0/2578042300 shutdown_connections 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.726+0000 7f45e1977640 1 -- 172.21.15.96:0/2578042300 wait complete. 2024-03-18T15:43:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 Processor -- start 2024-03-18T15:43:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 -- start start 2024-03-18T15:43:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 0x7f45dc196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10c380 con 0x7f45dc100700 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10c200 con 0x7f45dc1018f0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.727+0000 7f45e1977640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc10c500 con 0x7f45dc1022b0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42870/0 (socket says 172.21.15.96:42870) 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45daffd640 1 -- 172.21.15.96:0/2507984503 learned_addr learned my addr 172.21.15.96:0/2507984503 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45db7fe640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 0x7f45dc196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 -- 172.21.15.96:0/2507984503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 msgr2=0x7f45dc19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 -- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 msgr2=0x7f45dc196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 -- 172.21.15.96:0/2507984503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc19e580 con 0x7f45dc1018f0 2024-03-18T15:43:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45daffd640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45db7fe640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 0x7f45dc196ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009230 tx=0x7f45d002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.728+0000 7f45e0975640 1 -- 172.21.15.96:0/2507984503 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0041070 con 0x7f45dc1018f0 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.729+0000 7f45e0975640 1 -- 172.21.15.96:0/2507984503 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0004450 con 0x7f45dc1018f0 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.729+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc19e810 con 0x7f45dc1018f0 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.729+0000 7f45e0975640 1 -- 172.21.15.96:0/2507984503 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003df0 con 0x7f45dc1018f0 2024-03-18T15:43:44.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.729+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc19ed00 con 0x7f45dc1018f0 2024-03-18T15:43:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.731+0000 7f45e0975640 1 -- 172.21.15.96:0/2507984503 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f45d0044070 con 0x7f45dc1018f0 2024-03-18T15:43:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45e0975640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 0x7f45c40a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.734 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45e0975640 1 -- 172.21.15.96:0/2507984503 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f45d00f04b0 con 0x7f45dc1018f0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45daffd640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 0x7f45c40a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45daffd640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 0x7f45c40a4d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0058c0 tx=0x7f45cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45e41fd740 1 Processor -- start 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.732+0000 7f45e41fd740 1 -- start start 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570570760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e570575830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570578330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5704d5a40 con 0x55e570575ea0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e57047e920 con 0x55e570570d30 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45e41fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e5704d1ae0 con 0x55e570570340 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45daffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570570760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45da7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e570575830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570578330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45daffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570570760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48804/0 (socket says 172.21.15.96:48804) 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45daffd640 1 -- 172.21.15.96:0/1894112292 learned_addr learned my addr 172.21.15.96:0/1894112292 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 -- 172.21.15.96:0/1894112292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 msgr2=0x55e570570760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570570760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 -- 172.21.15.96:0/1894112292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 msgr2=0x55e570575830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e570575830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 -- 172.21.15.96:0/1894112292 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e570496460 con 0x55e570575ea0 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45da7fc640 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e570575830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.733+0000 7f45db7fe640 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570578330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005fd0 tx=0x7f45d4011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d4c27de30a725c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.734+0000 7f45c1ffb640 1 -- 172.21.15.96:0/1894112292 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004d20 con 0x55e570575ea0 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.734+0000 7f45c1ffb640 1 -- 172.21.15.96:0/1894112292 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d400f9a0 con 0x55e570575ea0 2024-03-18T15:43:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.734+0000 7f45c1ffb640 1 -- 172.21.15.96:0/1894112292 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4004470 con 0x55e570575ea0 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- 172.21.15.96:0/1894112292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 msgr2=0x55e570578330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570578330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005fd0 tx=0x7f45d4011b30 comp rx=0 tx=0).stop 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- 172.21.15.96:0/1894112292 shutdown_connections 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570578330 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e570575830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1894112292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570570760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- 172.21.15.96:0/1894112292 >> 172.21.15.96:0/1894112292 conn(0x55e57056b8b0 msgr2=0x55e57056bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- 172.21.15.96:0/1894112292 shutdown_connections 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- 172.21.15.96:0/1894112292 wait complete. 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.735+0000 7f45e41fd740 1 -- start start 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 Processor -- start 2024-03-18T15:43:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e5705836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570585cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e570578d50 con 0x55e570575ea0 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e570491d70 con 0x55e570570d30 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e570578ed0 con 0x55e570570340 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45e41fd740 10 client.0 _ll_register_callbacks cb 0x55e5701e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570585cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45daffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45da7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e5705836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45daffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48818/0 (socket says 172.21.15.96:48818) 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570585cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42886/0 (socket says 172.21.15.96:42886) 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45daffd640 1 -- 172.21.15.96:0/2965665474 learned_addr learned my addr 172.21.15.96:0/2965665474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.736+0000 7f45da7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e5705836d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42294/0 (socket says 172.21.15.96:42294) 2024-03-18T15:43:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 -- 172.21.15.96:0/2965665474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 msgr2=0x55e5705836d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e5705836d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 -- 172.21.15.96:0/2965665474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 msgr2=0x55e570585cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570585cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 -- 172.21.15.96:0/2965665474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5701ea210 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45daffd640 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00fc10 tx=0x7f45cc013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f530bbcd50343f8b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45baffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc01cda0 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d400f9a0 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f45cc004020 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45cc00fe80 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45e41fd740 10 client.26419 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.737+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5702b6620 con 0x55e570570340 2024-03-18T15:43:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d400f9a0 con 0x55e570570340 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f45cc027510 con 0x55e570570340 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 10 client.26419 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 20 client.26419 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e57058a890 0x55e57058cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 4 client.26419 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5704e2f80 con 0x55e57058a890 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e57058a890 0x55e57058cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45da7fc640 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e57058a890 0x55e57058cce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2a5e70dc97373 server_cookie=a2d2b28962466ace in_seq=0 out_seq=0 2024-03-18T15:43:44.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.738+0000 7f45baffd640 10 client.26419 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.739+0000 7f45baffd640 1 -- 172.21.15.96:0/2965665474 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e5704e2f80 con 0x55e57058a890 2024-03-18T15:43:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.739+0000 7f45baffd640 10 client.26419 handle_command_reply: tid=0 2024-03-18T15:43:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.739+0000 7f45e41fd740 1 client.26419 shutdown 2024-03-18T15:43:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 msgr2=0x55e570581180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00fc10 tx=0x7f45cc013040 comp rx=0 tx=0).stop 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 shutdown_connections 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e57058a890 0x55e57058cce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570575ea0 0x55e570585cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45baffd640 0 client.26419 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e5705836d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 --2- 172.21.15.96:0/2965665474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570570340 0x55e570581180 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 >> 172.21.15.96:0/2965665474 conn(0x55e57056b8b0 msgr2=0x55e57056abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 shutdown_connections 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 1 -- 172.21.15.96:0/2965665474 wait complete. 2024-03-18T15:43:44.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.740+0000 7f45e41fd740 20 client.26419 trim_cache size 0 max 16384 2024-03-18T15:43:44.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 Processor -- start 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 -- start start 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57056a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a6ad0 con 0x55e570570d30 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a86a0 con 0x55e570570340 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45e41fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a71a0 con 0x55e570575ea0 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45daffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45db7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57056a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45daffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:42310/0 (socket says 172.21.15.96:42310) 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42888/0 (socket says 172.21.15.96:42888) 2024-03-18T15:43:44.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45daffd640 1 -- 172.21.15.96:0/1584069775 learned_addr learned my addr 172.21.15.96:0/1584069775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 -- 172.21.15.96:0/1584069775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 msgr2=0x55e57056a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57056a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 -- 172.21.15.96:0/1584069775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 msgr2=0x55e5705925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45da7fc640 1 -- 172.21.15.96:0/1584069775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e57056c620 con 0x55e570570d30 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.756+0000 7f45daffd640 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.757+0000 7f45da7fc640 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f45d0006010 tx=0x7f45d00f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3ac61d06608834 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.757+0000 7f45c17fa640 1 -- 172.21.15.96:0/1584069775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d004b040 con 0x55e570570d30 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.757+0000 7f45c17fa640 1 -- 172.21.15.96:0/1584069775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d00c0f70 con 0x55e570570d30 2024-03-18T15:43:44.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.757+0000 7f45c17fa640 1 -- 172.21.15.96:0/1584069775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003710 con 0x55e570570d30 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- 172.21.15.96:0/1584069775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 msgr2=0x55e57056a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f45d0006010 tx=0x7f45d00f0040 comp rx=0 tx=0).stop 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- 172.21.15.96:0/1584069775 shutdown_connections 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57056a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570d30 0x55e57056a320 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 --2- 172.21.15.96:0/1584069775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570340 0x55e5705925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- 172.21.15.96:0/1584069775 >> 172.21.15.96:0/1584069775 conn(0x55e57056b8b0 msgr2=0x55e57056e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- 172.21.15.96:0/1584069775 shutdown_connections 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- 172.21.15.96:0/1584069775 wait complete. 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 -- start start 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.758+0000 7f45e41fd740 1 Processor -- start 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e57057e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57059cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a7fd0 con 0x55e570570340 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a7900 con 0x55e570570d30 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1a4ab0 con 0x55e570575ea0 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 10 client.0 _ll_register_callbacks cb 0x7f45cc00fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42892/0 (socket says 172.21.15.96:42892) 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 -- 172.21.15.96:0/3117014168 learned_addr learned my addr 172.21.15.96:0/3117014168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45da7fc640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e57057e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45db7fe640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57059cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 -- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 msgr2=0x55e57059cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57059cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 -- 172.21.15.96:0/3117014168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 msgr2=0x55e57057e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e57057e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 -- 172.21.15.96:0/3117014168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57059d5c0 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45daffd640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0279d0 tx=0x7f45cc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301478775f90ada9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45baffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc03e060 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f45cc027690 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.759+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d00c0f70 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f45cc036070 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 10 client.16578 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e5705ecbb0 con 0x55e570570340 2024-03-18T15:43:44.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d00c0f70 con 0x55e570570340 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f45cc004b80 con 0x55e570570340 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 10 client.16578 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 20 client.16578 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e5705ed580 0x55e5705ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 4 client.16578 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5704dd2d0 con 0x55e5705ed580 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.760+0000 7f45da7fc640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e5705ed580 0x55e5705ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.761+0000 7f45baffd640 10 client.16578 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:44.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.761+0000 7f45da7fc640 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e5705ed580 0x55e5705ef9a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2170d432676d86a server_cookie=d262c083591258ce in_seq=0 out_seq=0 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.771+0000 7f45baffd640 1 -- 172.21.15.96:0/3117014168 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5704dd2d0 con 0x55e5705ed580 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45baffd640 10 client.16578 handle_command_reply: tid=0 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 client.16578 shutdown 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 msgr2=0x55e57057e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0279d0 tx=0x7f45cc01c920 comp rx=0 tx=0).stop 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 shutdown_connections 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e570575ea0 0x55e57059cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e570570d30 0x55e57057e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e5705ed580 0x55e5705ef9a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 --2- 172.21.15.96:0/3117014168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e570570340 0x55e57057e270 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45baffd640 0 client.16578 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 >> 172.21.15.96:0/3117014168 conn(0x55e57056b8b0 msgr2=0x55e57056a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 shutdown_connections 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 1 -- 172.21.15.96:0/3117014168 wait complete. 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.772+0000 7f45e41fd740 20 client.16578 trim_cache size 0 max 16384 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:43:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:43:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 msgr2=0x7f45c40a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 0x7f45c40a4d60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0058c0 tx=0x7f45cc005f90 comp rx=0 tx=0).stop 2024-03-18T15:43:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 msgr2=0x7f45dc196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 0x7f45dc196ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009230 tx=0x7f45d002c650 comp rx=0 tx=0).stop 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 shutdown_connections 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f45c40a28a0 0x7f45c40a4d60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45dc1022b0 0x7f45dc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45dc1018f0 0x7f45dc196ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 --2- 172.21.15.96:0/2507984503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45dc100700 0x7f45dc196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 >> 172.21.15.96:0/2507984503 conn(0x7f45dc0fbc70 msgr2=0x7f45dc104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 shutdown_connections 2024-03-18T15:43:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:44.773+0000 7f45e1977640 1 -- 172.21.15.96:0/2507984503 wait complete. 2024-03-18T15:43:44.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:44 smithi137 ceph-mon[35868]: pgmap v461: 97 pgs: 97 active+clean; 434 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-03-18T15:43:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:44 smithi193 ceph-mon[36702]: pgmap v461: 97 pgs: 97 active+clean; 434 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-03-18T15:43:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:44 smithi096 ceph-mon[31930]: pgmap v461: 97 pgs: 97 active+clean; 434 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 84 op/s 2024-03-18T15:43:46.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:46 smithi137 ceph-mon[35868]: pgmap v462: 97 pgs: 97 active+clean; 434 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-03-18T15:43:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:46 smithi193 ceph-mon[36702]: pgmap v462: 97 pgs: 97 active+clean; 434 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-03-18T15:43:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:46 smithi096 ceph-mon[31930]: pgmap v462: 97 pgs: 97 active+clean; 434 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-03-18T15:43:48.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:48 smithi137 ceph-mon[35868]: pgmap v463: 97 pgs: 97 active+clean; 433 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-03-18T15:43:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:48 smithi193 ceph-mon[36702]: pgmap v463: 97 pgs: 97 active+clean; 433 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-03-18T15:43:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:48 smithi096 ceph-mon[31930]: pgmap v463: 97 pgs: 97 active+clean; 433 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-03-18T15:43:50.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:50 smithi137 ceph-mon[35868]: pgmap v464: 97 pgs: 97 active+clean; 438 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-03-18T15:43:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:50 smithi193 ceph-mon[36702]: pgmap v464: 97 pgs: 97 active+clean; 438 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-03-18T15:43:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:50 smithi096 ceph-mon[31930]: pgmap v464: 97 pgs: 97 active+clean; 438 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 108 op/s 2024-03-18T15:43:51.329 INFO:tasks.workunit.client.0.smithi096.stdout: 91 1677039 431 1240768 265 980020 994 2024-03-18T15:43:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:52 smithi193 ceph-mon[36702]: pgmap v465: 97 pgs: 97 active+clean; 438 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-03-18T15:43:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:52 smithi096 ceph-mon[31930]: pgmap v465: 97 pgs: 97 active+clean; 438 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-03-18T15:43:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:52 smithi137 ceph-mon[35868]: pgmap v465: 97 pgs: 97 active+clean; 438 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 103 op/s 2024-03-18T15:43:54.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:54 smithi137 ceph-mon[35868]: pgmap v466: 97 pgs: 97 active+clean; 465 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-18T15:43:54.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:54 smithi193 ceph-mon[36702]: pgmap v466: 97 pgs: 97 active+clean; 465 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-18T15:43:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:54 smithi096 ceph-mon[31930]: pgmap v466: 97 pgs: 97 active+clean; 465 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 277 op/s 2024-03-18T15:43:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:56 smithi193 ceph-mon[36702]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 401 op/s 2024-03-18T15:43:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:56 smithi096 ceph-mon[31930]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 401 op/s 2024-03-18T15:43:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:56 smithi137 ceph-mon[35868]: pgmap v467: 97 pgs: 97 active+clean; 493 MiB data, 933 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.5 MiB/s wr, 401 op/s 2024-03-18T15:43:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.201+0000 7f7911157640 1 -- 172.21.15.96:0/1222620460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c078d60 msgr2=0x7f790c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.201+0000 7f7911157640 1 --2- 172.21.15.96:0/1222620460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c078d60 0x7f790c07b1f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7904009ca0 tx=0x7f790402c120 comp rx=0 tx=0).stop 2024-03-18T15:43:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 -- 172.21.15.96:0/1222620460 shutdown_connections 2024-03-18T15:43:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 --2- 172.21.15.96:0/1222620460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c078d60 0x7f790c07b1f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 --2- 172.21.15.96:0/1222620460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0763b0 0x7f790c078820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 --2- 172.21.15.96:0/1222620460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c0734d0 0x7f790c0738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 -- 172.21.15.96:0/1222620460 >> 172.21.15.96:0/1222620460 conn(0x7f790c06e650 msgr2=0x7f790c070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 -- 172.21.15.96:0/1222620460 shutdown_connections 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.206+0000 7f7911157640 1 -- 172.21.15.96:0/1222620460 wait complete. 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 Processor -- start 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 -- start start 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 0x7f790c12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c078d60 0x7f790c131520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f790c07ead0 con 0x7f790c0734d0 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f790c07e950 con 0x7f790c078d60 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.207+0000 7f7911157640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f790c07ec50 con 0x7f790c0763b0 2024-03-18T15:43:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33596/0 (socket says 172.21.15.96:33596) 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f7910956640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c078d60 0x7f790c131520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790bfff640 1 -- 172.21.15.96:0/2194868363 learned_addr learned my addr 172.21.15.96:0/2194868363 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 0x7f790c12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 -- 172.21.15.96:0/2194868363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c078d60 msgr2=0x7f790c131520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c078d60 0x7f790c131520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 -- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 msgr2=0x7f790c12ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 -- 172.21.15.96:0/2194868363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c131de0 con 0x7f790c0763b0 2024-03-18T15:43:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790bfff640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.208+0000 7f790b7fe640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 0x7f790c12ef70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f790000c970 tx=0x7f790000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.216+0000 7f79097fa640 1 -- 172.21.15.96:0/2194868363 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900007bb0 con 0x7f790c0763b0 2024-03-18T15:43:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.216+0000 7f79097fa640 1 -- 172.21.15.96:0/2194868363 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7900007d50 con 0x7f790c0763b0 2024-03-18T15:43:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.216+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c1320d0 con 0x7f790c0763b0 2024-03-18T15:43:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.216+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1325f0 con 0x7f790c0763b0 2024-03-18T15:43:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.218+0000 7f79097fa640 1 -- 172.21.15.96:0/2194868363 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790000f450 con 0x7f790c0763b0 2024-03-18T15:43:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.221+0000 7f79097fa640 1 -- 172.21.15.96:0/2194868363 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f790000f6b0 con 0x7f790c0763b0 2024-03-18T15:43:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.224+0000 7f7914fa4740 1 Processor -- start 2024-03-18T15:43:58.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 -- start start 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a7030 0x561d828a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d82806a40 con 0x561d828a1ec0 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561d827af920 con 0x561d828a14d0 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7914fa4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d82802ae0 con 0x561d828a7030 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7910956640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7910956640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:39190/0 (socket says 172.21.15.96:39190) 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.225+0000 7f7910956640 1 -- 172.21.15.96:0/2795266546 learned_addr learned my addr 172.21.15.96:0/2795266546 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f790b7fe640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a7030 0x561d828a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f790bfff640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 -- 172.21.15.96:0/2795266546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a7030 msgr2=0x561d828a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a7030 0x561d828a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 -- 172.21.15.96:0/2795266546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 msgr2=0x561d828a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 -- 172.21.15.96:0/2795266546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d827c7460 con 0x561d828a14d0 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f790bfff640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f7910956640 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7904005e30 tx=0x7f79040323f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b05f1fe9a8e308e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.226+0000 7f79097fa640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 0x7f78f40a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.227+0000 7f790bfff640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 0x7f78f40a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.227+0000 7f79097fa640 1 -- 172.21.15.96:0/2194868363 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f79000cf600 con 0x7f790c0763b0 2024-03-18T15:43:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.227+0000 7f790bfff640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 0x7f78f40a4e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x561d828a22c0 tx=0x7f78fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.234+0000 7f78fa7fc640 1 -- 172.21.15.96:0/2795266546 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904043070 con 0x561d828a14d0 2024-03-18T15:43:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.234+0000 7f78fa7fc640 1 -- 172.21.15.96:0/2795266546 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790402ccd0 con 0x561d828a14d0 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 -- 172.21.15.96:0/2795266546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 msgr2=0x561d828a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7904005e30 tx=0x7f79040323f0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 -- 172.21.15.96:0/2795266546 shutdown_connections 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a7030 0x561d828a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 --2- 172.21.15.96:0/2795266546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a14d0 0x561d828a18f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 -- 172.21.15.96:0/2795266546 >> 172.21.15.96:0/2795266546 conn(0x561d8289ca40 msgr2=0x561d8289ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 -- 172.21.15.96:0/2795266546 shutdown_connections 2024-03-18T15:43:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.235+0000 7f7914fa4740 1 -- 172.21.15.96:0/2795266546 wait complete. 2024-03-18T15:43:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.236+0000 7f7914fa4740 1 -- start start 2024-03-18T15:43:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.236+0000 7f7914fa4740 1 Processor -- start 2024-03-18T15:43:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 0x561d828abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d828a9fd0 con 0x561d828a14d0 2024-03-18T15:43:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561d827c2d70 con 0x561d828a7030 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561d828aa150 con 0x561d828a1ec0 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7914fa4740 10 client.0 _ll_register_callbacks cb 0x561d8251aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f790bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f790bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33626/0 (socket says 172.21.15.96:33626) 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f790bfff640 1 -- 172.21.15.96:0/3912909370 learned_addr learned my addr 172.21.15.96:0/3912909370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f790b7fe640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 0x561d828abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.237+0000 7f7910956640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 -- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 msgr2=0x561d828abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 0x561d828abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 -- 172.21.15.96:0/3912909370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 msgr2=0x561d828abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 -- 172.21.15.96:0/3912909370 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d8251b210 con 0x561d828a7030 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f790bfff640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f790b7fe640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 0x561d828abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:43:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f7910956640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b3030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7904009bf0 tx=0x7f7904032700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6779b7d3b2421005 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.238+0000 7f78ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790404e070 con 0x561d828a7030 2024-03-18T15:43:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7904004560 con 0x561d828a7030 2024-03-18T15:43:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561d82813f80 con 0x561d828a7030 2024-03-18T15:43:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f790405a050 con 0x561d828a7030 2024-03-18T15:43:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f7914fa4740 10 client.26384 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d825e7620 con 0x561d828a7030 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561d82813f80 con 0x561d828a7030 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f790402dce0 con 0x561d828a7030 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.239+0000 7f7914fa4740 10 client.26384 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f7914fa4740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f7914fa4740 20 client.26384 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d828b7520 0x561d828b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f7914fa4740 4 client.26384 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d8280e2d0 con 0x561d828b7520 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f790bfff640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d828b7520 0x561d828b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.243 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f78ef7fe640 10 client.26384 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.240+0000 7f790bfff640 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d828b7520 0x561d828b9910 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf778231802dc865 server_cookie=1f9c5f63a80420fc in_seq=0 out_seq=0 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.241+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3912909370 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561d8280e2d0 con 0x561d828b7520 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.241+0000 7f78ef7fe640 10 client.26384 handle_command_reply: tid=0 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 client.26384 shutdown 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 msgr2=0x561d828b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b3030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7904009bf0 tx=0x7f7904032700 comp rx=0 tx=0).stop 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 shutdown_connections 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b3030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a1ec0 0x561d828abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d828b7520 0x561d828b9910 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3912909370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f78ef7fe640 0 client.26384 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 >> 172.21.15.96:0/3912909370 conn(0x561d8289ca40 msgr2=0x561d8289bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 shutdown_connections 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.242+0000 7f7914fa4740 1 -- 172.21.15.96:0/3912909370 wait complete. 2024-03-18T15:43:58.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.243+0000 7f7914fa4740 20 client.26384 trim_cache size 0 max 16384 2024-03-18T15:43:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.274+0000 7f7914fa4740 1 Processor -- start 2024-03-18T15:43:58.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.274+0000 7f7914fa4740 1 -- start start 2024-03-18T15:43:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f790bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f790bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48772/0 (socket says 172.21.15.96:48772) 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f7910956640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f7910956640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33642/0 (socket says 172.21.15.96:33642) 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:39204/0 (socket says 172.21.15.96:39204) 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.275+0000 7f790bfff640 1 -- 172.21.15.96:0/4186685225 learned_addr learned my addr 172.21.15.96:0/4186685225 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000e500 con 0x561d828a7030 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78f00100d0 con 0x561d828a1ec0 2024-03-18T15:43:58.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000ebd0 con 0x561d828a14d0 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790b7fe640 1 -- 172.21.15.96:0/4186685225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 msgr2=0x561d828abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790b7fe640 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790b7fe640 1 -- 172.21.15.96:0/4186685225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 msgr2=0x561d828abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790b7fe640 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790b7fe640 1 -- 172.21.15.96:0/4186685225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d828a1110 con 0x561d828a1ec0 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f790bfff640 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.276+0000 7f7910956640 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.277+0000 7f790b7fe640 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79000058c0 tx=0x7f790000f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eafef26f63c4e5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.277+0000 7f78f9ffb640 1 -- 172.21.15.96:0/4186685225 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790001d040 con 0x561d828a1ec0 2024-03-18T15:43:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.277+0000 7f78f9ffb640 1 -- 172.21.15.96:0/4186685225 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790009bfc0 con 0x561d828a1ec0 2024-03-18T15:43:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.277+0000 7f78f9ffb640 1 -- 172.21.15.96:0/4186685225 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900096b30 con 0x561d828a1ec0 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 msgr2=0x561d828b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f79000058c0 tx=0x7f790000f000 comp rx=0 tx=0).stop 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 shutdown_connections 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a7030 0x561d828abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a1ec0 0x561d828b2ff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 --2- 172.21.15.96:0/4186685225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 >> 172.21.15.96:0/4186685225 conn(0x561d8289ca40 msgr2=0x561d828a3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 shutdown_connections 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.278+0000 7f7914fa4740 1 -- 172.21.15.96:0/4186685225 wait complete. 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 -- start start 2024-03-18T15:43:58.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 Processor -- start 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000fa00 con 0x561d828a1ec0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000f330 con 0x561d828a7030 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.279+0000 7f7914fa4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78f000c4e0 con 0x561d828a14d0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48788/0 (socket says 172.21.15.96:48788) 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33658/0 (socket says 172.21.15.96:33658) 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790bfff640 1 -- 172.21.15.96:0/3960795795 learned_addr learned my addr 172.21.15.96:0/3960795795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7914fa4740 10 client.0 _ll_register_callbacks cb 0x7f7900004800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:58.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 -- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 msgr2=0x561d828b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 -- 172.21.15.96:0/3960795795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 msgr2=0x561d828b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 -- 172.21.15.96:0/3960795795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d828b21b0 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f790b7fe640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7910956640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b1bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f790402c660 tx=0x7f7904005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5807c5cb6c77b038 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f78ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904057070 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790009bfc0 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.280+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7904048650 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.281+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f790404e040 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.281+0000 7f7914fa4740 10 client.26390 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.281+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561d8291d7a0 con 0x561d828a7030 2024-03-18T15:43:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.281+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790009bfc0 con 0x561d828a7030 2024-03-18T15:43:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f790402dce0 con 0x561d828a7030 2024-03-18T15:43:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 10 client.26390 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 20 client.26390 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d8291e030 0x561d82920450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 4 client.26390 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d8278c770 con 0x561d8291e030 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f790bfff640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d8291e030 0x561d82920450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f790bfff640 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d8291e030 0x561d82920450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a107758bdd510f30 server_cookie=1aad2263cea579ec in_seq=0 out_seq=0 2024-03-18T15:43:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.282+0000 7f78ef7fe640 10 client.26390 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:43:58.826 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f78ef7fe640 1 -- 172.21.15.96:0/3960795795 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d8278c770 con 0x561d8291e030 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f78ef7fe640 10 client.26390 handle_command_reply: tid=0 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 client.26390 shutdown 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 msgr2=0x561d828b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b1bc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f790402c660 tx=0x7f7904005cf0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 shutdown_connections 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d828a7030 0x561d828b1bc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561d8291e030 0x561d82920450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d828a1ec0 0x561d828b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f78ef7fe640 0 client.26390 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 --2- 172.21.15.96:0/3960795795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561d828a14d0 0x561d828b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 >> 172.21.15.96:0/3960795795 conn(0x561d8289ca40 msgr2=0x561d828aba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 shutdown_connections 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 1 -- 172.21.15.96:0/3960795795 wait complete. 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.864+0000 7f7914fa4740 20 client.26390 trim_cache size 0 max 16384 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:43:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 msgr2=0x7f78f40a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 0x7f78f40a4e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x561d828a22c0 tx=0x7f78fc005f90 comp rx=0 tx=0).stop 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 msgr2=0x7f790c12ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 0x7f790c12ef70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f790000c970 tx=0x7f790000ce30 comp rx=0 tx=0).stop 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 shutdown_connections 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f78f40a29c0 0x7f78f40a4e80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f790c078d60 0x7f790c131520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f790c0763b0 0x7f790c12ef70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 --2- 172.21.15.96:0/2194868363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f790c0734d0 0x7f790c12ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 >> 172.21.15.96:0/2194868363 conn(0x7f790c06e650 msgr2=0x7f790c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 shutdown_connections 2024-03-18T15:43:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:43:58.865+0000 7f7911157640 1 -- 172.21.15.96:0/2194868363 wait complete. 2024-03-18T15:43:58.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:43:58 smithi193 ceph-mon[36702]: pgmap v468: 97 pgs: 97 active+clean; 499 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 438 op/s 2024-03-18T15:43:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:43:58 smithi096 ceph-mon[31930]: pgmap v468: 97 pgs: 97 active+clean; 499 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 438 op/s 2024-03-18T15:43:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 -- 172.21.15.96:0/749654693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 msgr2=0x7f74080ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 --2- 172.21.15.96:0/749654693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f74080ffee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009890 tx=0x7f73f802bb70 comp rx=0 tx=0).stop 2024-03-18T15:43:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 -- 172.21.15.96:0/749654693 shutdown_connections 2024-03-18T15:43:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 --2- 172.21.15.96:0/749654693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 0x7f7408109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 --2- 172.21.15.96:0/749654693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 0x7f7408102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 --2- 172.21.15.96:0/749654693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f74080ffee0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 -- 172.21.15.96:0/749654693 >> 172.21.15.96:0/749654693 conn(0x7f74080fb3d0 msgr2=0x7f74080fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 -- 172.21.15.96:0/749654693 shutdown_connections 2024-03-18T15:43:58.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.948+0000 7f740d113640 1 -- 172.21.15.96:0/749654693 wait complete. 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 Processor -- start 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 -- start start 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 0x7f74081980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 0x7f740819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.949+0000 7f740d113640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f740810d470 con 0x7f7408102b30 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f740d113640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f740810d2f0 con 0x7f74080ffac0 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f740d113640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f740810d5f0 con 0x7f74081034f0 2024-03-18T15:43:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:39224/0 (socket says 172.21.15.96:39224) 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 -- 172.21.15.96:0/608232872 learned_addr learned my addr 172.21.15.96:0/608232872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f740659c640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 0x7f74081980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f740759e640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 0x7f740819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 -- 172.21.15.96:0/608232872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 msgr2=0x7f740819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 0x7f740819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 -- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 msgr2=0x7f74081980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 0x7f74081980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 -- 172.21.15.96:0/608232872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819f790 con 0x7f74080ffac0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.950+0000 7f7406d9d640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73f80349e0 tx=0x7f73f8034760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.951+0000 7f73effff640 1 -- 172.21.15.96:0/608232872 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8039040 con 0x7f74080ffac0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.951+0000 7f73effff640 1 -- 172.21.15.96:0/608232872 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f8042070 con 0x7f74080ffac0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.951+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819fa20 con 0x7f74080ffac0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.951+0000 7f73effff640 1 -- 172.21.15.96:0/608232872 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8032cb0 con 0x7f74080ffac0 2024-03-18T15:43:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.951+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740819ff10 con 0x7f74080ffac0 2024-03-18T15:43:58.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f73effff640 1 -- 172.21.15.96:0/608232872 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f73f802c510 con 0x7f74080ffac0 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 Processor -- start 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 -- start start 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 0x55d175f5e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d175eb7500 con 0x55d175f5c0e0 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d175e5fbe0 con 0x55d175f52b60 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740fdce740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d175eb35a0 con 0x55d175f52170 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48808/0 (socket says 172.21.15.96:48808) 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740759e640 1 -- 172.21.15.96:0/1293620266 learned_addr learned my addr 172.21.15.96:0/1293620266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f7406d9d640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.954+0000 7f740659c640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 0x55d175f5e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 -- 172.21.15.96:0/1293620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 msgr2=0x55d175f57660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f57660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 -- 172.21.15.96:0/1293620266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 msgr2=0x55d175f5e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f73effff640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 0x7f73f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 0x55d175f5e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 -- 172.21.15.96:0/1293620266 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d175e734b0 con 0x55d175f52170 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740659c640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 0x55d175f5e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f7406d9d640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 0x7f73f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f740759e640 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7400009fd0 tx=0x7f74000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6229a0b8ad488bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f73effff640 1 -- 172.21.15.96:0/608232872 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f73f80f02e0 con 0x7f74080ffac0 2024-03-18T15:43:58.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f73ed7fa640 1 -- 172.21.15.96:0/1293620266 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400004d20 con 0x55d175f52170 2024-03-18T15:43:58.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f73ed7fa640 1 -- 172.21.15.96:0/1293620266 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004050 con 0x55d175f52170 2024-03-18T15:43:58.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.955+0000 7f73ed7fa640 1 -- 172.21.15.96:0/1293620266 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400005260 con 0x55d175f52170 2024-03-18T15:43:58.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 -- 172.21.15.96:0/1293620266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 msgr2=0x55d175f52590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f7406d9d640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 0x7f73f00a4cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ee120 tx=0x7f73f80f3dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7400009fd0 tx=0x7f74000118d0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 -- 172.21.15.96:0/1293620266 shutdown_connections 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f5c0e0 0x55d175f5e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f57660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 --2- 172.21.15.96:0/1293620266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f52590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 -- 172.21.15.96:0/1293620266 >> 172.21.15.96:0/1293620266 conn(0x55d175f4d6e0 msgr2=0x55d175f4dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 -- 172.21.15.96:0/1293620266 shutdown_connections 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.956+0000 7f740fdce740 1 -- 172.21.15.96:0/1293620266 wait complete. 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 -- start start 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 Processor -- start 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f52b60 0x55d175f55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f5c0e0 0x55d175f56b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d175f58050 con 0x55d175f52b60 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d175e73160 con 0x55d175f52170 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d175f581d0 con 0x55d175f5c0e0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740fdce740 10 client.0 _ll_register_callbacks cb 0x55d175e57150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:39242/0 (socket says 172.21.15.96:39242) 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740759e640 1 -- 172.21.15.96:0/811198949 learned_addr learned my addr 172.21.15.96:0/811198949 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740659c640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f52b60 0x55d175f55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.957+0000 7f740759e640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f5c0e0 0x55d175f56b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 -- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f5c0e0 msgr2=0x55d175f56b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f5c0e0 0x55d175f56b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 -- 172.21.15.96:0/811198949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f52b60 msgr2=0x55d175f55b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f52b60 0x55d175f55b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 -- 172.21.15.96:0/811198949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d175c957d0 con 0x55d175f52170 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740759e640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f740001b040 tx=0x7f7400009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b7538998fad0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:58.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f73eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740003c020 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f740000f4e0 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.958+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400004050 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7400047050 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 10 client.26402 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d175bcb3f0 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004050 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f74000197d0 con 0x55d175f52170 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 10 client.26402 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 20 client.26402 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f70780 0x55d175f72b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 4 client.26402 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.959+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d175ec4a40 con 0x55d175f70780 2024-03-18T15:43:58.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.960+0000 7f740659c640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f70780 0x55d175f72b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.960+0000 7f740659c640 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f70780 0x55d175f72b70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93ffc0baa7c7f37 server_cookie=89dd4e849928d8c2 in_seq=0 out_seq=0 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.960+0000 7f73eaff5640 10 client.26402 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f73eaff5640 1 -- 172.21.15.96:0/811198949 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d175ec4a40 con 0x55d175f70780 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f73eaff5640 10 client.26402 handle_command_reply: tid=0 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 client.26402 shutdown 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 msgr2=0x55d175f555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f740001b040 tx=0x7f7400009c80 comp rx=0 tx=0).stop 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 shutdown_connections 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f5c0e0 0x55d175f56b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f70780 0x55d175f72b70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f52b60 0x55d175f55b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f73eaff5640 0 client.26402 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 --2- 172.21.15.96:0/811198949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f555c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.961+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 >> 172.21.15.96:0/811198949 conn(0x55d175f4d6e0 msgr2=0x55d175f4c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.962+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 shutdown_connections 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.962+0000 7f740fdce740 1 -- 172.21.15.96:0/811198949 wait complete. 2024-03-18T15:43:58.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.962+0000 7f740fdce740 20 client.26402 trim_cache size 0 max 16384 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.998+0000 7f740fdce740 1 Processor -- start 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 -- start start 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f54370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f4c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f4c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7408071db0 con 0x55d175f4d6e0 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f74081a0140 con 0x55d175f52b60 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740fdce740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7408072480 con 0x55d175f52170 2024-03-18T15:43:59.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f54370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f4c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f54370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33690/0 (socket says 172.21.15.96:33690) 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740759e640 1 -- 172.21.15.96:0/908319933 learned_addr learned my addr 172.21.15.96:0/908319933 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740759e640 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f4c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740659c640 1 -- 172.21.15.96:0/908319933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 msgr2=0x55d175f4c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740659c640 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f4c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740659c640 1 -- 172.21.15.96:0/908319933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 msgr2=0x55d175f54370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:58.999+0000 7f740659c640 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f54370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.000+0000 7f740659c640 1 -- 172.21.15.96:0/908319933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d175b8b0f0 con 0x55d175f52170 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.000+0000 7f740659c640 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f4c070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d175f733e0 tx=0x7f73fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce60de1539e3d3b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.000+0000 7f73ecff9640 1 -- 172.21.15.96:0/908319933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01e130 con 0x55d175f52170 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.000+0000 7f73ecff9640 1 -- 172.21.15.96:0/908319933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73fc022020 con 0x55d175f52170 2024-03-18T15:43:59.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.000+0000 7f73ecff9640 1 -- 172.21.15.96:0/908319933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc01dd10 con 0x55d175f52170 2024-03-18T15:43:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 -- 172.21.15.96:0/908319933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 msgr2=0x55d175f4c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f4c070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d175f733e0 tx=0x7f73fc0046c0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 -- 172.21.15.96:0/908319933 shutdown_connections 2024-03-18T15:43:59.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52b60 0x55d175f4c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52170 0x55d175f4c070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 --2- 172.21.15.96:0/908319933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f54370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 -- 172.21.15.96:0/908319933 >> 172.21.15.96:0/908319933 conn(0x55d175f588e0 msgr2=0x55d175f53c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 -- 172.21.15.96:0/908319933 shutdown_connections 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.001+0000 7f740fdce740 1 -- 172.21.15.96:0/908319933 wait complete. 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 -- start start 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 Processor -- start 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f6abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52b60 0x55d175f6bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74080732b0 con 0x55d175f4d6e0 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7408072be0 con 0x55d175f52170 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f740806fd90 con 0x55d175f52b60 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740fdce740 10 client.0 _ll_register_callbacks cb 0x7f73f80f3c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33704/0 (socket says 172.21.15.96:33704) 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f6abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740759e640 1 -- 172.21.15.96:0/1830964722 learned_addr learned my addr 172.21.15.96:0/1830964722 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.002+0000 7f740759e640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52b60 0x55d175f6bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 -- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52b60 msgr2=0x55d175f6bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52b60 0x55d175f6bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 -- 172.21.15.96:0/1830964722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 msgr2=0x55d175f6abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f6abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 -- 172.21.15.96:0/1830964722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d175e7e9d0 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740659c640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740759e640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7400002230 tx=0x7f7400009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf39085ed8954e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740003c040 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7400019430 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73fc022020 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7400019630 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740fdce740 10 client.16608 fetch_fsmap learned FSMap version 32 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d175f6d2d0 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73fc022020 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.003+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f740000f4e0 con 0x55d175f4d6e0 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 10 client.16608 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 20 client.16608 populate_metadata read hostname 'smithi096' 2024-03-18T15:43:59.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f5c0e0 0x55d175fd9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 4 client.16608 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d175ebed90 con 0x55d175f5c0e0 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740659c640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f5c0e0 0x55d175fd9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f73eaff5640 10 client.16608 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.004+0000 7f740659c640 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f5c0e0 0x55d175fd9100 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5b708849077f5e server_cookie=98bec0bb43c610ec in_seq=0 out_seq=0 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f73eaff5640 1 -- 172.21.15.96:0/1830964722 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d175ebed90 con 0x55d175f5c0e0 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f73eaff5640 10 client.16608 handle_command_reply: tid=0 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 client.16608 shutdown 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 msgr2=0x55d175f6a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7400002230 tx=0x7f7400009fd0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 shutdown_connections 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d175f52b60 0x55d175f6bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d175f52170 0x55d175f6abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d175f5c0e0 0x55d175fd9100 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 --2- 172.21.15.96:0/1830964722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d175f4d6e0 0x55d175f6a6a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f73eaff5640 0 client.16608 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 >> 172.21.15.96:0/1830964722 conn(0x55d175f588e0 msgr2=0x55d175f50940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 shutdown_connections 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 1 -- 172.21.15.96:0/1830964722 wait complete. 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.005+0000 7f740fdce740 20 client.16608 trim_cache size 0 max 16384 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:43:59.012 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "201eed03-ac93-4b1d-816a-172ace6d3466": { 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "201eed03-ac93-4b1d-816a-172ace6d3466", 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:43:59.013 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:43:59.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 msgr2=0x7f73f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:59.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 0x7f73f00a4cc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ee120 tx=0x7f73f80f3dc0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 msgr2=0x7f7408197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:43:59.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f73f80349e0 tx=0x7f73f8034760 comp rx=0 tx=0).stop 2024-03-18T15:43:59.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 shutdown_connections 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73f00a2800 0x7f73f00a4cc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74081034f0 0x7f740819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7408102b30 0x7f74081980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 --2- 172.21.15.96:0/608232872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74080ffac0 0x7f7408197b70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 >> 172.21.15.96:0/608232872 conn(0x7f74080fb3d0 msgr2=0x7f7408105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 shutdown_connections 2024-03-18T15:43:59.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:43:59.006+0000 7f740d113640 1 -- 172.21.15.96:0/608232872 wait complete. 2024-03-18T15:43:59.027 INFO:tasks.cephfs.filesystem:scrub status for tag:201eed03-ac93-4b1d-816a-172ace6d3466 - {'path': '/', 'tag': '201eed03-ac93-4b1d-816a-172ace6d3466', 'options': 'recursive,force'} 2024-03-18T15:43:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:43:58 smithi137 ceph-mon[35868]: pgmap v468: 97 pgs: 97 active+clean; 499 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 438 op/s 2024-03-18T15:44:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:00 smithi193 ceph-mon[36702]: pgmap v469: 97 pgs: 97 active+clean; 498 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-03-18T15:44:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:00 smithi096 ceph-mon[31930]: pgmap v469: 97 pgs: 97 active+clean; 498 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-03-18T15:44:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:00 smithi137 ceph-mon[35868]: pgmap v469: 97 pgs: 97 active+clean; 498 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.2 MiB/s wr, 464 op/s 2024-03-18T15:44:01.330 INFO:tasks.workunit.client.0.smithi096.stdout: 94 1097350 243 815794 210 638638 660 2024-03-18T15:44:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:02 smithi193 ceph-mon[36702]: pgmap v470: 97 pgs: 97 active+clean; 501 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 448 op/s 2024-03-18T15:44:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:02 smithi096 ceph-mon[31930]: pgmap v470: 97 pgs: 97 active+clean; 501 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 448 op/s 2024-03-18T15:44:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:02 smithi137 ceph-mon[35868]: pgmap v470: 97 pgs: 97 active+clean; 501 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.0 MiB/s wr, 448 op/s 2024-03-18T15:44:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:04 smithi193 ceph-mon[36702]: pgmap v471: 97 pgs: 97 active+clean; 503 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:44:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:04 smithi096 ceph-mon[31930]: pgmap v471: 97 pgs: 97 active+clean; 503 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:44:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:04 smithi137 ceph-mon[35868]: pgmap v471: 97 pgs: 97 active+clean; 503 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 462 op/s 2024-03-18T15:44:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:06 smithi193 ceph-mon[36702]: pgmap v472: 97 pgs: 97 active+clean; 502 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-03-18T15:44:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:06 smithi096 ceph-mon[31930]: pgmap v472: 97 pgs: 97 active+clean; 502 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-03-18T15:44:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:06 smithi137 ceph-mon[35868]: pgmap v472: 97 pgs: 97 active+clean; 502 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-03-18T15:44:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:08 smithi193 ceph-mon[36702]: pgmap v473: 97 pgs: 97 active+clean; 505 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-03-18T15:44:08.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:08 smithi096 ceph-mon[31930]: pgmap v473: 97 pgs: 97 active+clean; 505 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-03-18T15:44:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:08 smithi137 ceph-mon[35868]: pgmap v473: 97 pgs: 97 active+clean; 505 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-03-18T15:44:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:10 smithi193 ceph-mon[36702]: pgmap v474: 97 pgs: 97 active+clean; 503 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-03-18T15:44:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:10 smithi096 ceph-mon[31930]: pgmap v474: 97 pgs: 97 active+clean; 503 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-03-18T15:44:11.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:10 smithi137 ceph-mon[35868]: pgmap v474: 97 pgs: 97 active+clean; 503 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-03-18T15:44:11.330 INFO:tasks.workunit.client.0.smithi096.stdout: 99 1550394 377 1157258 289 879633 1020 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 -- 172.21.15.96:0/2553319729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b41094d0 msgr2=0x7f08b410e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2553319729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b41094d0 0x7f08b410e140 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009f00 tx=0x7f08ac02c3a0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 -- 172.21.15.96:0/2553319729 shutdown_connections 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2553319729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b41094d0 0x7f08b410e140 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2553319729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08b40733f0 0x7f08b4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2553319729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b4072a90 0x7f08b4072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.677+0000 7f08b3fff640 1 -- 172.21.15.96:0/2553319729 >> 172.21.15.96:0/2553319729 conn(0x7f08b406e650 msgr2=0x7f08b4070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.689+0000 7f08b3fff640 1 -- 172.21.15.96:0/2553319729 shutdown_connections 2024-03-18T15:44:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.690+0000 7f08b3fff640 1 -- 172.21.15.96:0/2553319729 wait complete. 2024-03-18T15:44:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 Processor -- start 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 -- start start 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08b4072a90 0x7f08b4114e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 0x7f08b4115370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08b411c170 con 0x7f08b40733f0 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08b411bff0 con 0x7f08b41094d0 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b3fff640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08b411c2f0 con 0x7f08b4072a90 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:40996/0 (socket says 172.21.15.96:40996) 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.691+0000 7f08b37fe640 1 -- 172.21.15.96:0/2719428324 learned_addr learned my addr 172.21.15.96:0/2719428324 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b27fc640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 0x7f08b4115370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b37fe640 1 -- 172.21.15.96:0/2719428324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08b4072a90 msgr2=0x7f08b4114e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b37fe640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08b4072a90 0x7f08b4114e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b37fe640 1 -- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 msgr2=0x7f08b4115370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b37fe640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 0x7f08b4115370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b37fe640 1 -- 172.21.15.96:0/2719428324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b41184a0 con 0x7f08b41094d0 2024-03-18T15:44:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.692+0000 7f08b27fc640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 0x7f08b4115370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.693+0000 7f08b37fe640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08ac032a70 tx=0x7f08ac003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.715+0000 7f08b885f640 1 -- 172.21.15.96:0/2719428324 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0338d0 con 0x7f08b41094d0 2024-03-18T15:44:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.715+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b41b6e20 con 0x7f08b41094d0 2024-03-18T15:44:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.715+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41b7280 con 0x7f08b41094d0 2024-03-18T15:44:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.715+0000 7f08b885f640 1 -- 172.21.15.96:0/2719428324 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ac033a70 con 0x7f08b41094d0 2024-03-18T15:44:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.715+0000 7f08b885f640 1 -- 172.21.15.96:0/2719428324 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac03fa40 con 0x7f08b41094d0 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 Processor -- start 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 -- start start 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca654d0 0x5594dca658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5594dc9caa40 con 0x5594dca6b030 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5594dc973920 con 0x5594dca65ec0 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08bc453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594dc9c6ae0 con 0x5594dca654d0 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:40998/0 (socket says 172.21.15.96:40998) 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08b27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.725+0000 7f08b2ffd640 1 -- 172.21.15.96:0/218678301 learned_addr learned my addr 172.21.15.96:0/218678301 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 -- 172.21.15.96:0/218678301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca654d0 msgr2=0x5594dca658f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca654d0 0x5594dca658f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 -- 172.21.15.96:0/218678301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 msgr2=0x5594dca6d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 -- 172.21.15.96:0/218678301 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dc98b460 con 0x5594dca65ec0 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b27fc640 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:44:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.726+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08a800bb10 tx=0x7f08a800d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e822229291cf3c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.730+0000 7f08b885f640 1 -- 172.21.15.96:0/2719428324 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f08ac03fbe0 con 0x7f08b41094d0 2024-03-18T15:44:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.731+0000 7f08b885f640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 0x7f089c0a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.731+0000 7f08b27fc640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 0x7f089c0a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.731+0000 7f08b885f640 1 -- 172.21.15.96:0/2719428324 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f08ac0f31f0 con 0x7f08b41094d0 2024-03-18T15:44:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.732+0000 7f08b27fc640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 0x7f089c0a4ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5594dca66720 tx=0x7f08a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:12.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.753+0000 7f08a1ffb640 1 -- 172.21.15.96:0/218678301 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8016070 con 0x5594dca65ec0 2024-03-18T15:44:12.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.753+0000 7f08a1ffb640 1 -- 172.21.15.96:0/218678301 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8005500 con 0x5594dca65ec0 2024-03-18T15:44:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.754+0000 7f08bc453740 1 -- 172.21.15.96:0/218678301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 msgr2=0x5594dca6a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.754+0000 7f08bc453740 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f08a800bb10 tx=0x7f08a800d2e0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.755 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 -- 172.21.15.96:0/218678301 shutdown_connections 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca65ec0 0x5594dca6a9c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 --2- 172.21.15.96:0/218678301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca654d0 0x5594dca658f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 -- 172.21.15.96:0/218678301 >> 172.21.15.96:0/218678301 conn(0x5594dca60a40 msgr2=0x5594dca60e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 -- 172.21.15.96:0/218678301 shutdown_connections 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 -- 172.21.15.96:0/218678301 wait complete. 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.755+0000 7f08bc453740 1 -- start start 2024-03-18T15:44:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 Processor -- start 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 0x5594dca6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5594dca6e030 con 0x5594dca65ec0 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5594dc986d70 con 0x5594dca654d0 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594dca6e1b0 con 0x5594dca6b030 2024-03-18T15:44:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.756+0000 7f08bc453740 10 client.0 _ll_register_callbacks cb 0x5594dc6dea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48194/0 (socket says 172.21.15.96:48194) 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 -- 172.21.15.96:0/4247903977 learned_addr learned my addr 172.21.15.96:0/4247903977 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b27fc640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 0x5594dca6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 -- 172.21.15.96:0/4247903977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 msgr2=0x5594dca6f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 -- 172.21.15.96:0/4247903977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 msgr2=0x5594dca6fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 0x5594dca6fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.757+0000 7f08b37fe640 1 -- 172.21.15.96:0/4247903977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dc6df210 con 0x5594dca6b030 2024-03-18T15:44:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.758+0000 7f08b27fc640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 0x5594dca6fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:44:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.758+0000 7f08b37fe640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0bbed0 tx=0x7f08ac0bf990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc25632cc4cda2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.807+0000 7f0896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:44:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.807+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0d7070 con 0x5594dca6b030 2024-03-18T15:44:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.807+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f08ac0de220 con 0x5594dca6b030 2024-03-18T15:44:12.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.807+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a8005500 con 0x5594dca6b030 2024-03-18T15:44:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.813+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08ac032df0 con 0x5594dca6b030 2024-03-18T15:44:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.814+0000 7f08bc453740 10 client.26464 fetch_fsmap learned FSMap version 32 2024-03-18T15:44:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594dc7ab620 con 0x5594dca6b030 2024-03-18T15:44:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8005500 con 0x5594dca6b030 2024-03-18T15:44:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f08ac04f030 con 0x5594dca6b030 2024-03-18T15:44:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 10 client.26464 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:44:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 20 client.26464 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dca7b870 0x5594dca7dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 4 client.26464 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:12.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.819+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594dc9d7f80 con 0x5594dca7b870 2024-03-18T15:44:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.820+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dca7b870 0x5594dca7dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:12.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.820+0000 7f0896ffd640 10 client.26464 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.820+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dca7b870 0x5594dca7dcb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf9ad3163ce2d3f server_cookie=ff24f56a2a484823 in_seq=0 out_seq=0 2024-03-18T15:44:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.821+0000 7f0896ffd640 1 -- 172.21.15.96:0/4247903977 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5594dc9d7f80 con 0x5594dca7b870 2024-03-18T15:44:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.821+0000 7f0896ffd640 10 client.26464 handle_command_reply: tid=0 2024-03-18T15:44:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.822+0000 7f08bc453740 1 client.26464 shutdown 2024-03-18T15:44:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.823+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 msgr2=0x5594dca76d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.823+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08ac0bbed0 tx=0x7f08ac0bf990 comp rx=0 tx=0).stop 2024-03-18T15:44:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 shutdown_connections 2024-03-18T15:44:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca6b030 0x5594dca76d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dca7b870 0x5594dca7dcb0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca65ec0 0x5594dca6fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 --2- 172.21.15.96:0/4247903977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 >> 172.21.15.96:0/4247903977 conn(0x5594dca60a40 msgr2=0x5594dca5fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f0896ffd640 0 client.26464 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 shutdown_connections 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 1 -- 172.21.15.96:0/4247903977 wait complete. 2024-03-18T15:44:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:12.837+0000 7f08bc453740 20 client.26464 trim_cache size 0 max 16384 2024-03-18T15:44:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:12 smithi193 ceph-mon[36702]: pgmap v475: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T15:44:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:12 smithi096 ceph-mon[31930]: pgmap v475: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T15:44:13.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.008+0000 7f08bc453740 1 Processor -- start 2024-03-18T15:44:13.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.009+0000 7f08bc453740 1 -- start start 2024-03-18T15:44:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.010+0000 7f08bc453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.010+0000 7f08bc453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.010+0000 7f08bc453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.010+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.010+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41024/0 (socket says 172.21.15.96:41024) 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48200/0 (socket says 172.21.15.96:48200) 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 -- 172.21.15.96:0/2372889688 learned_addr learned my addr 172.21.15.96:0/2372889688 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f089800e620 con 0x5594dca6b030 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08980101f0 con 0x5594dca654d0 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f089800ecf0 con 0x5594dca65ec0 2024-03-18T15:44:13.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b37fe640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 -- 172.21.15.96:0/2372889688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 msgr2=0x5594dca76d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 -- 172.21.15.96:0/2372889688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 msgr2=0x5594dca6eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b2ffd640 1 -- 172.21.15.96:0/2372889688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594dca617b0 con 0x5594dca654d0 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b27fc640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.011+0000 7f08b37fe640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.012+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08a800b8a0 tx=0x7f08a800dd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db22e7cd34b89b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.035+0000 7f08a17fa640 1 -- 172.21.15.96:0/2372889688 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8016040 con 0x5594dca654d0 2024-03-18T15:44:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.036+0000 7f08a17fa640 1 -- 172.21.15.96:0/2372889688 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a800de90 con 0x5594dca654d0 2024-03-18T15:44:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.036+0000 7f08a17fa640 1 -- 172.21.15.96:0/2372889688 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8005400 con 0x5594dca654d0 2024-03-18T15:44:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.038+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 msgr2=0x5594dca6e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.038+0000 7f08bc453740 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08a800b8a0 tx=0x7f08a800dd20 comp rx=0 tx=0).stop 2024-03-18T15:44:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.038+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 shutdown_connections 2024-03-18T15:44:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.039+0000 7f08bc453740 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dca6eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.039+0000 7f08bc453740 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca76d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.039+0000 7f08bc453740 1 --2- 172.21.15.96:0/2372889688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca6e630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.039+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 >> 172.21.15.96:0/2372889688 conn(0x5594dca60a40 msgr2=0x5594dca67100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.040+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 shutdown_connections 2024-03-18T15:44:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.040+0000 7f08bc453740 1 -- 172.21.15.96:0/2372889688 wait complete. 2024-03-18T15:44:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.040+0000 7f08bc453740 1 -- start start 2024-03-18T15:44:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.041+0000 7f08bc453740 1 Processor -- start 2024-03-18T15:44:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.041+0000 7f08bc453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca73b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca73b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca73b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:41034/0 (socket says 172.21.15.96:41034) 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b27fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48214/0 (socket says 172.21.15.96:48214) 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42156/0 (socket says 172.21.15.96:42156) 2024-03-18T15:44:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f089800fb20 con 0x5594dca6b030 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f089800f450 con 0x5594dca654d0 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08b2ffd640 1 -- 172.21.15.96:0/1933663076 learned_addr learned my addr 172.21.15.96:0/1933663076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f089800c600 con 0x5594dca65ec0 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.042+0000 7f08bc453740 10 client.0 _ll_register_callbacks cb 0x7f08ac005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 -- 172.21.15.96:0/1933663076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 msgr2=0x5594dca73b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:13.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca73b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 -- 172.21.15.96:0/1933663076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 msgr2=0x5594dcadfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 -- 172.21.15.96:0/1933663076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594dcae0120 con 0x5594dca65ec0 2024-03-18T15:44:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b37fe640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.043+0000 7f08b27fc640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5594dca70ca0 tx=0x7f08a40046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ad354cc8598d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.063+0000 7f0896ffd640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:44:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.063+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4028070 con 0x5594dca65ec0 2024-03-18T15:44:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.063+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f08a4024470 con 0x5594dca65ec0 2024-03-18T15:44:13.064 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.063+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a800de90 con 0x5594dca65ec0 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f08a4032050 con 0x5594dca65ec0 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f08bc453740 10 client.26470 fetch_fsmap learned FSMap version 32 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5594dcae1370 con 0x5594dca65ec0 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a401d040 con 0x5594dca65ec0 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7f08a4022040 con 0x5594dca65ec0 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.064+0000 7f08bc453740 10 client.26470 fetch_fsmap finished waiting for FSMap version 32 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08bc453740 10 client.26470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08bc453740 20 client.26470 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dcae1d50 0x5594dcae4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08bc453740 4 client.26470 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f08a800de90 con 0x5594dcae1d50 2024-03-18T15:44:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.065+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dcae1d50 0x5594dcae4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.066+0000 7f08b2ffd640 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dcae1d50 0x5594dcae4170 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166111f2244130c server_cookie=8f1523622d39bea3 in_seq=0 out_seq=0 2024-03-18T15:44:13.067 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.067+0000 7f0896ffd640 10 client.26470 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:13.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:12 smithi137 ceph-mon[35868]: pgmap v475: 97 pgs: 97 active+clean; 508 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f0896ffd640 1 -- 172.21.15.96:0/1933663076 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f08a800de90 con 0x5594dcae1d50 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f0896ffd640 10 client.26470 handle_command_reply: tid=0 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 client.26470 shutdown 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 msgr2=0x5594dca74050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x5594dca70ca0 tx=0x7f08a40046e0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 shutdown_connections 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5594dcae1d50 0x5594dcae4170 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5594dca6b030 0x5594dcadfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594dca65ec0 0x5594dca74050 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 --2- 172.21.15.96:0/1933663076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5594dca654d0 0x5594dca73b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 >> 172.21.15.96:0/1933663076 conn(0x5594dca60a40 msgr2=0x5594dca63ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f0896ffd640 0 client.26470 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 shutdown_connections 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.286+0000 7f08bc453740 1 -- 172.21.15.96:0/1933663076 wait complete. 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.287+0000 7f08bc453740 20 client.26470 trim_cache size 0 max 16384 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:44:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.308+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 msgr2=0x7f089c0a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.308+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 0x7f089c0a4ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5594dca66720 tx=0x7f08a4008040 comp rx=0 tx=0).stop 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.308+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 msgr2=0x7f08b41b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f08ac032a70 tx=0x7f08ac003f10 comp rx=0 tx=0).stop 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 shutdown_connections 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f089c0a2a00 0x7f089c0a4ec0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08b41094d0 0x7f08b41b6880 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08b40733f0 0x7f08b4115370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 --2- 172.21.15.96:0/2719428324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08b4072a90 0x7f08b4114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.309+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 >> 172.21.15.96:0/2719428324 conn(0x7f08b406e650 msgr2=0x7f08b410a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.315+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 shutdown_connections 2024-03-18T15:44:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:13.316+0000 7f08b3fff640 1 -- 172.21.15.96:0/2719428324 wait complete. 2024-03-18T15:44:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:14 smithi193 ceph-mon[36702]: pgmap v476: 97 pgs: 97 active+clean; 506 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T15:44:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:14 smithi096 ceph-mon[31930]: pgmap v476: 97 pgs: 97 active+clean; 506 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T15:44:15.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:14 smithi137 ceph-mon[35868]: pgmap v476: 97 pgs: 97 active+clean; 506 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-03-18T15:44:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:16 smithi193 ceph-mon[36702]: pgmap v477: 97 pgs: 97 active+clean; 511 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T15:44:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:16 smithi096 ceph-mon[31930]: pgmap v477: 97 pgs: 97 active+clean; 511 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T15:44:17.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:16 smithi137 ceph-mon[35868]: pgmap v477: 97 pgs: 97 active+clean; 511 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-03-18T15:44:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:18 smithi193 ceph-mon[36702]: pgmap v478: 97 pgs: 97 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-03-18T15:44:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:18 smithi096 ceph-mon[31930]: pgmap v478: 97 pgs: 97 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-03-18T15:44:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:18 smithi137 ceph-mon[35868]: pgmap v478: 97 pgs: 97 active+clean; 519 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 199 op/s 2024-03-18T15:44:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:20 smithi193 ceph-mon[36702]: pgmap v479: 97 pgs: 97 active+clean; 546 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-03-18T15:44:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:20 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:44:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:20 smithi096 ceph-mon[31930]: pgmap v479: 97 pgs: 97 active+clean; 546 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-03-18T15:44:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:20 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:44:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:20 smithi137 ceph-mon[35868]: pgmap v479: 97 pgs: 97 active+clean; 546 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-03-18T15:44:21.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:20 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:44:21.330 INFO:tasks.workunit.client.0.smithi096.stdout: 103 1467602 286 1107398 145 841216 685 2024-03-18T15:44:21.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:21 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:44:21.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:21 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:44:22.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:21 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:44:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:22 smithi193 ceph-mon[36702]: pgmap v480: 97 pgs: 97 active+clean; 556 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 456 op/s 2024-03-18T15:44:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:22 smithi096 ceph-mon[31930]: pgmap v480: 97 pgs: 97 active+clean; 556 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 456 op/s 2024-03-18T15:44:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:22 smithi137 ceph-mon[35868]: pgmap v480: 97 pgs: 97 active+clean; 556 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.3 MiB/s wr, 456 op/s 2024-03-18T15:44:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:24 smithi193 ceph-mon[36702]: pgmap v481: 97 pgs: 97 active+clean; 565 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 475 op/s 2024-03-18T15:44:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:24 smithi096 ceph-mon[31930]: pgmap v481: 97 pgs: 97 active+clean; 565 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 475 op/s 2024-03-18T15:44:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:24 smithi137 ceph-mon[35868]: pgmap v481: 97 pgs: 97 active+clean; 565 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.7 MiB/s wr, 475 op/s 2024-03-18T15:44:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- 172.21.15.96:0/521097345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4100710 msgr2=0x7faac4100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 --2- 172.21.15.96:0/521097345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4100710 0x7faac4100b30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faab4009900 tx=0x7faab402bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- 172.21.15.96:0/521097345 shutdown_connections 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 --2- 172.21.15.96:0/521097345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac41022c0 0x7faac4108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 --2- 172.21.15.96:0/521097345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac4101900 0x7faac4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 --2- 172.21.15.96:0/521097345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4100710 0x7faac4100b30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- 172.21.15.96:0/521097345 >> 172.21.15.96:0/521097345 conn(0x7faac40fbc80 msgr2=0x7faac40fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- 172.21.15.96:0/521097345 shutdown_connections 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- 172.21.15.96:0/521097345 wait complete. 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 Processor -- start 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.541+0000 7faac9d32640 1 -- start start 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac41022c0 0x7faac419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faac410c380 con 0x7faac4101900 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faac410c200 con 0x7faac4100710 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac9d32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faac410c500 con 0x7faac41022c0 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38076/0 (socket says 172.21.15.96:38076) 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 -- 172.21.15.96:0/3618997161 learned_addr learned my addr 172.21.15.96:0/3618997161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38876/0 (socket says 172.21.15.96:38876) 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac3fff640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac41022c0 0x7faac419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 -- 172.21.15.96:0/3618997161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac41022c0 msgr2=0x7faac419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac41022c0 0x7faac419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 -- 172.21.15.96:0/3618997161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 msgr2=0x7faac4196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 -- 172.21.15.96:0/3618997161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac419e580 con 0x7faac4101900 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac37fe640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faab800e9f0 tx=0x7faab800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.542+0000 7faac0ff9640 1 -- 172.21.15.96:0/3618997161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab800cd80 con 0x7faac4101900 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.543+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac419e870 con 0x7faac4101900 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.543+0000 7faac0ff9640 1 -- 172.21.15.96:0/3618997161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8005500 con 0x7faac4101900 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.543+0000 7faac0ff9640 1 -- 172.21.15.96:0/3618997161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8010690 con 0x7faac4101900 2024-03-18T15:44:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.543+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac419ed70 con 0x7faac4101900 2024-03-18T15:44:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faac0ff9640 1 -- 172.21.15.96:0/3618997161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7faab8010830 con 0x7faac4101900 2024-03-18T15:44:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 Processor -- start 2024-03-18T15:44:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 -- start start 2024-03-18T15:44:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faac0ff9640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 0x7faaac0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a66cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a678a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b2a5d1a40 con 0x558b2a66c650 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b2a57a920 con 0x558b2a66d9c0 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faacc9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b2a5cdae0 con 0x558b2a676620 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faac37fe640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 0x7faaac0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.545+0000 7faac3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38886/0 (socket says 172.21.15.96:38886) 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 -- 172.21.15.96:0/3462615793 learned_addr learned my addr 172.21.15.96:0/3462615793 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac0ff9640 1 -- 172.21.15.96:0/3618997161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faab80cee60 con 0x7faac4101900 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a66cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a678a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 -- 172.21.15.96:0/3462615793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 msgr2=0x558b2a678a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a678a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 -- 172.21.15.96:0/3462615793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 msgr2=0x558b2a66cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a66cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 -- 172.21.15.96:0/3462615793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2a592460 con 0x558b2a66d9c0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac37fe640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 0x7faaac0a4d60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faac3fff640 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faabc005f40 tx=0x7faabc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182cb495dd554fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faaa9ffb640 1 -- 172.21.15.96:0/3462615793 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc011070 con 0x558b2a66d9c0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faaa9ffb640 1 -- 172.21.15.96:0/3462615793 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc00fd60 con 0x558b2a66d9c0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.546+0000 7faaa9ffb640 1 -- 172.21.15.96:0/3462615793 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc010be0 con 0x558b2a66d9c0 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- 172.21.15.96:0/3462615793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 msgr2=0x558b2a66c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faabc005f40 tx=0x7faabc009b80 comp rx=0 tx=0).stop 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- 172.21.15.96:0/3462615793 shutdown_connections 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a678a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a66cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3462615793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a66c110 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- 172.21.15.96:0/3462615793 >> 172.21.15.96:0/3462615793 conn(0x558b2a6678b0 msgr2=0x558b2a667cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- 172.21.15.96:0/3462615793 shutdown_connections 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- 172.21.15.96:0/3462615793 wait complete. 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.547+0000 7faacc9f3740 1 -- start start 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 Processor -- start 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a681fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b2a679250 con 0x558b2a66c650 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b2a58dd70 con 0x558b2a66d9c0 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558b2a6793d0 con 0x558b2a676620 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faacc9f3740 10 client.0 _ll_register_callbacks cb 0x558b2a2e5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38086/0 (socket says 172.21.15.96:38086) 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38892/0 (socket says 172.21.15.96:38892) 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 -- 172.21.15.96:0/3951609604 learned_addr learned my addr 172.21.15.96:0/3951609604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a681fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 -- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 msgr2=0x558b2a681fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a681fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 -- 172.21.15.96:0/3951609604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 msgr2=0x558b2a67af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac37fe640 1 -- 172.21.15.96:0/3951609604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2a2e6210 con 0x558b2a66c650 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.548+0000 7faac3fff640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a681fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faac37fe640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faab4009900 tx=0x7faab4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f701c0714c062fb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab404e070 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faab4048440 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc00fd60 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faab405a050 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 10 client.16635 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558b2a3b2620 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faabc00fd60 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7faab40347a0 con 0x558b2a66c650 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 10 client.16635 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 20 client.16635 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a6868c0 0x558b2a688cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 4 client.16635 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.549+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b2a5def80 con 0x558b2a6868c0 2024-03-18T15:44:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.550+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a6868c0 0x558b2a688cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.550+0000 7faaa2ffd640 10 client.16635 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.550+0000 7faac2ffd640 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a6868c0 0x558b2a688cd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f28796a97d8c24 server_cookie=4c2edeca818f5ac7 in_seq=0 out_seq=0 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faaa2ffd640 1 -- 172.21.15.96:0/3951609604 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558b2a5def80 con 0x558b2a6868c0 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faaa2ffd640 10 client.16635 handle_command_reply: tid=0 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 client.16635 shutdown 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 msgr2=0x558b2a67aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faab4009900 tx=0x7faab4009260 comp rx=0 tx=0).stop 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 shutdown_connections 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a676620 0x558b2a681fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66d9c0 0x558b2a67af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a6868c0 0x558b2a688cd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 --2- 172.21.15.96:0/3951609604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67aa40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faaa2ffd640 0 client.16635 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:26.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 >> 172.21.15.96:0/3951609604 conn(0x558b2a6678b0 msgr2=0x558b2a666bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 shutdown_connections 2024-03-18T15:44:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 1 -- 172.21.15.96:0/3951609604 wait complete. 2024-03-18T15:44:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.551+0000 7faacc9f3740 20 client.16635 trim_cache size 0 max 16384 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 Processor -- start 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 -- start start 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 0x558b2a679770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a67aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a6b00 con 0x558b2a6678b0 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a8700 con 0x558b2a66c650 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faacc9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a71d0 con 0x558b2a66d9c0 2024-03-18T15:44:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faac37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 0x558b2a679770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38900/0 (socket says 172.21.15.96:38900) 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.566+0000 7faac2ffd640 1 -- 172.21.15.96:0/2092862587 learned_addr learned my addr 172.21.15.96:0/2092862587 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac3fff640 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a67aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 -- 172.21.15.96:0/2092862587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 msgr2=0x558b2a67aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a67aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 -- 172.21.15.96:0/2092862587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 msgr2=0x558b2a679770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 0x558b2a679770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 -- 172.21.15.96:0/2092862587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2a673200 con 0x558b2a66c650 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac37fe640 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 0x558b2a679770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faac2ffd640 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faab809b760 tx=0x7faab8086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b2456ff712d3c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faaa97fa640 1 -- 172.21.15.96:0/2092862587 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80af070 con 0x558b2a66c650 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faaa97fa640 1 -- 172.21.15.96:0/2092862587 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab809bed0 con 0x558b2a66c650 2024-03-18T15:44:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.567+0000 7faaa97fa640 1 -- 172.21.15.96:0/2092862587 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80a1020 con 0x558b2a66c650 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- 172.21.15.96:0/2092862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 msgr2=0x558b2a681f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faab809b760 tx=0x7faab8086040 comp rx=0 tx=0).stop 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- 172.21.15.96:0/2092862587 shutdown_connections 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a67aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a66c650 0x558b2a681f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 --2- 172.21.15.96:0/2092862587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a6678b0 0x558b2a679770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- 172.21.15.96:0/2092862587 >> 172.21.15.96:0/2092862587 conn(0x558b2a676240 msgr2=0x558b2a66ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- 172.21.15.96:0/2092862587 shutdown_connections 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- 172.21.15.96:0/2092862587 wait complete. 2024-03-18T15:44:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.568+0000 7faacc9f3740 1 -- start start 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 Processor -- start 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 0x558b2a67fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a680f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a8030 con 0x558b2a66c650 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a7960 con 0x558b2a6678b0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faac41a4ae0 con 0x558b2a66d9c0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faacc9f3740 10 client.0 _ll_register_callbacks cb 0x7faab4041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 0x558b2a67fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38104/0 (socket says 172.21.15.96:38104) 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac2ffd640 1 -- 172.21.15.96:0/1774753128 learned_addr learned my addr 172.21.15.96:0/1774753128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac3fff640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a680f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 -- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 msgr2=0x558b2a680f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a680f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 -- 172.21.15.96:0/1774753128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 msgr2=0x558b2a67ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac37fe640 1 -- 172.21.15.96:0/1774753128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2a681520 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.569+0000 7faac2ffd640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faac37fe640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 0x558b2a67fa30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faab402c7b0 tx=0x7faab4009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4e414190061e57 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab404e040 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab809bed0 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faab4034640 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faab4034840 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 10 client.26444 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558b2a6ecfc0 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab809bed0 con 0x558b2a6678b0 2024-03-18T15:44:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7faab4053560 con 0x558b2a6678b0 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 10 client.26444 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 10 client.26444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 20 client.26444 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a676620 0x558b2a6ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 4 client.26444 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.570+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b2a5d92d0 con 0x558b2a676620 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.571+0000 7faac2ffd640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a676620 0x558b2a6ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.571+0000 7faaa2ffd640 10 client.26444 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:26.571+0000 7faac2ffd640 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a676620 0x558b2a6ef940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce68944cae2167d3 server_cookie=ff2c3a4d64a25ac8 in_seq=0 out_seq=0 2024-03-18T15:44:26.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:26 smithi193 ceph-mon[36702]: pgmap v482: 97 pgs: 97 active+clean; 564 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 473 op/s 2024-03-18T15:44:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:26 smithi096 ceph-mon[31930]: pgmap v482: 97 pgs: 97 active+clean; 564 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 473 op/s 2024-03-18T15:44:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:26 smithi137 ceph-mon[35868]: pgmap v482: 97 pgs: 97 active+clean; 564 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 473 op/s 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faaa2ffd640 1 -- 172.21.15.96:0/1774753128 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b2a5d92d0 con 0x558b2a676620 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faaa2ffd640 10 client.26444 handle_command_reply: tid=0 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 client.26444 shutdown 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 msgr2=0x558b2a67fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 0x558b2a67fa30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faab402c7b0 tx=0x7faab4009590 comp rx=0 tx=0).stop 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 shutdown_connections 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2a66d9c0 0x558b2a680f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558b2a676620 0x558b2a6ef940 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b2a66c650 0x558b2a67ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faaa2ffd640 0 client.26444 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 --2- 172.21.15.96:0/1774753128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b2a6678b0 0x558b2a67fa30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 >> 172.21.15.96:0/1774753128 conn(0x558b2a676240 msgr2=0x558b2a66b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 shutdown_connections 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 1 -- 172.21.15.96:0/1774753128 wait complete. 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.216+0000 7faacc9f3740 20 client.26444 trim_cache size 0 max 16384 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:44:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 msgr2=0x7faaac0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 0x7faaac0a4d60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab4005fd0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 msgr2=0x7faac4196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faab800e9f0 tx=0x7faab800edf0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 shutdown_connections 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faaac0a28a0 0x7faaac0a4d60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faac41022c0 0x7faac419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faac4101900 0x7faac4196ea0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 --2- 172.21.15.96:0/3618997161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faac4100710 0x7faac4196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 >> 172.21.15.96:0/3618997161 conn(0x7faac40fbc80 msgr2=0x7faac4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 shutdown_connections 2024-03-18T15:44:28.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:28.217+0000 7faac9d32640 1 -- 172.21.15.96:0/3618997161 wait complete. 2024-03-18T15:44:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:28 smithi096 ceph-mon[31930]: pgmap v483: 97 pgs: 97 active+clean; 562 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-18T15:44:29.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:44:29.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:44:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:28 smithi137 ceph-mon[35868]: pgmap v483: 97 pgs: 97 active+clean; 562 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-18T15:44:29.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 -- 172.21.15.96:0/281204549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 msgr2=0x7f143c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 --2- 172.21.15.96:0/281204549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c101d50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1430009900 tx=0x7f143002bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 -- 172.21.15.96:0/281204549 shutdown_connections 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 --2- 172.21.15.96:0/281204549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 --2- 172.21.15.96:0/281204549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c101d50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 --2- 172.21.15.96:0/281204549 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 -- 172.21.15.96:0/281204549 >> 172.21.15.96:0/281204549 conn(0x7f143c0fbc70 msgr2=0x7f143c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 -- 172.21.15.96:0/281204549 shutdown_connections 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.137+0000 7f1442234640 1 -- 172.21.15.96:0/281204549 wait complete. 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 Processor -- start 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 -- start start 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f143c10c340 con 0x7f143c1018d0 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f143c10c1c0 con 0x7f143c1006e0 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f1442234640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f143c10c4c0 con 0x7f143c102290 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.138+0000 7f143affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38110/0 (socket says 172.21.15.96:38110) 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c0694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38920/0 (socket says 172.21.15.96:38920) 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143affd640 1 -- 172.21.15.96:0/2778416047 learned_addr learned my addr 172.21.15.96:0/2778416047 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 -- 172.21.15.96:0/2778416047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 msgr2=0x7f143c0694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c0694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 -- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 msgr2=0x7f143c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 -- 172.21.15.96:0/2778416047 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c19f760 con 0x7f143c102290 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143affd640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f143bfff640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c19eea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1434009fd0 tx=0x7f143400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f1438ff9640 1 -- 172.21.15.96:0/2778416047 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143400ce20 con 0x7f143c102290 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c19fa50 con 0x7f143c102290 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f1438ff9640 1 -- 172.21.15.96:0/2778416047 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1434005020 con 0x7f143c102290 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f1438ff9640 1 -- 172.21.15.96:0/2778416047 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434010720 con 0x7f143c102290 2024-03-18T15:44:29.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.139+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c19ff70 con 0x7f143c102290 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.142+0000 7f1444e1c740 1 Processor -- start 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.142+0000 7f1438ff9640 1 -- 172.21.15.96:0/2778416047 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1434005540 con 0x7f143c102290 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 -- start start 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40c170 0x55692b40c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 0x55692b414160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55692b371500 con 0x55692b411cd0 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55692b319be0 con 0x55692b40c170 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1444e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55692b36d5a0 con 0x55692b40cb60 2024-03-18T15:44:29.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 0x55692b414160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:59150/0 (socket says 172.21.15.96:59150) 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 -- 172.21.15.96:0/45804797 learned_addr learned my addr 172.21.15.96:0/45804797 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143affd640 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40c170 0x55692b40c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1438ff9640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 0x7f14240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 -- 172.21.15.96:0/45804797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40c170 msgr2=0x55692b40c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40c170 0x55692b40c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 -- 172.21.15.96:0/45804797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 msgr2=0x55692b414160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 0x55692b414160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143bfff640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 0x7f14240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143b7fe640 1 -- 172.21.15.96:0/45804797 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692b32d4b0 con 0x55692b40cb60 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f143affd640 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 0x55692b414160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.143+0000 7f1438ff9640 1 -- 172.21.15.96:0/2778416047 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f14340cf170 con 0x7f143c102290 2024-03-18T15:44:29.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.144+0000 7f143b7fe640 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f142c009ca0 tx=0x7f142c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b8234d2d4abfe55 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.144+0000 7f143bfff640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 0x7f14240a4cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f143409ba70 tx=0x7f143406d760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.144+0000 7f1421ffb640 1 -- 172.21.15.96:0/45804797 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c004220 con 0x55692b40cb60 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.144+0000 7f1421ffb640 1 -- 172.21.15.96:0/45804797 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c0043c0 con 0x55692b40cb60 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.144+0000 7f1421ffb640 1 -- 172.21.15.96:0/45804797 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c019890 con 0x55692b40cb60 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- 172.21.15.96:0/45804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 msgr2=0x55692b411660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f142c009ca0 tx=0x7f142c011810 comp rx=0 tx=0).stop 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- 172.21.15.96:0/45804797 shutdown_connections 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b411cd0 0x55692b414160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40cb60 0x55692b411660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 --2- 172.21.15.96:0/45804797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40c170 0x55692b40c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- 172.21.15.96:0/45804797 >> 172.21.15.96:0/45804797 conn(0x55692b4076e0 msgr2=0x55692b407ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- 172.21.15.96:0/45804797 shutdown_connections 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- 172.21.15.96:0/45804797 wait complete. 2024-03-18T15:44:29.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 -- start start 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.145+0000 7f1444e1c740 1 Processor -- start 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b416690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b41dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55692b414c80 con 0x55692b40cb60 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55692b32d160 con 0x55692b411cd0 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55692b414e00 con 0x55692b40c170 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f1444e1c740 10 client.0 _ll_register_callbacks cb 0x55692b311150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38118/0 (socket says 172.21.15.96:38118) 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 -- 172.21.15.96:0/1277355912 learned_addr learned my addr 172.21.15.96:0/1277355912 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:29.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143affd640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b416690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143affd640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b41dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 -- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 msgr2=0x55692b416690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b416690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 -- 172.21.15.96:0/1277355912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 msgr2=0x55692b41dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b41dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 -- 172.21.15.96:0/1277355912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692b14f7d0 con 0x55692b40cb60 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f142c009700 tx=0x7f142c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c115d567fc873e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f141affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.146+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c028070 con 0x55692b40cb60 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f142c003800 con 0x55692b40cb60 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142c0043c0 con 0x55692b40cb60 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f142c034050 con 0x55692b40cb60 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 10 client.16653 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:29.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55692b0853f0 con 0x55692b40cb60 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c0043c0 con 0x55692b40cb60 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f142c004990 con 0x55692b40cb60 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 10 client.16653 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 20 client.16653 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b421f70 0x55692b4243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 4 client.16653 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692b37ea40 con 0x55692b421f70 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.147+0000 7f143affd640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b421f70 0x55692b4243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.148+0000 7f143affd640 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b421f70 0x55692b4243b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dee7f06b1f1ad61 server_cookie=55d189e78824b20b in_seq=0 out_seq=0 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.148+0000 7f141affd640 10 client.16653 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f141affd640 1 -- 172.21.15.96:0/1277355912 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55692b37ea40 con 0x55692b421f70 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f141affd640 10 client.16653 handle_command_reply: tid=0 2024-03-18T15:44:29.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 client.16653 shutdown 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 msgr2=0x55692b416bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f142c009700 tx=0x7f142c009f90 comp rx=0 tx=0).stop 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 shutdown_connections 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b41dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b421f70 0x55692b4243b0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b416bd0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1277355912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b416690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f141affd640 0 client.16653 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 >> 172.21.15.96:0/1277355912 conn(0x55692b4076e0 msgr2=0x55692b406900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 shutdown_connections 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 1 -- 172.21.15.96:0/1277355912 wait complete. 2024-03-18T15:44:29.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.149+0000 7f1444e1c740 20 client.16653 trim_cache size 0 max 16384 2024-03-18T15:44:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:28 smithi193 ceph-mon[36702]: pgmap v483: 97 pgs: 97 active+clean; 562 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 5.2 MiB/s wr, 462 op/s 2024-03-18T15:44:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 Processor -- start 2024-03-18T15:44:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 -- start start 2024-03-18T15:44:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b406070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b406af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1a8e00 con 0x55692b40cb60 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1aaa00 con 0x55692b411cd0 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f1444e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1a94d0 con 0x55692b40c170 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f143b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38130/0 (socket says 172.21.15.96:38130) 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f143b7fe640 1 -- 172.21.15.96:0/2384588785 learned_addr learned my addr 172.21.15.96:0/2384588785 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f143affd640 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b406070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.167+0000 7f143affd640 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b406af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 -- 172.21.15.96:0/2384588785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 msgr2=0x55692b406070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b406070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 -- 172.21.15.96:0/2384588785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 msgr2=0x55692b406af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b406af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 -- 172.21.15.96:0/2384588785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692b0450f0 con 0x55692b40cb60 2024-03-18T15:44:29.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f143b7fe640 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f142c011850 tx=0x7f142c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a2ace0737f3111 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f14217fa640 1 -- 172.21.15.96:0/2384588785 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c028040 con 0x55692b40cb60 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f14217fa640 1 -- 172.21.15.96:0/2384588785 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c031070 con 0x55692b40cb60 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.168+0000 7f14217fa640 1 -- 172.21.15.96:0/2384588785 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c004b40 con 0x55692b40cb60 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- 172.21.15.96:0/2384588785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 msgr2=0x55692b4065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f142c011850 tx=0x7f142c005db0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- 172.21.15.96:0/2384588785 shutdown_connections 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b411cd0 0x55692b406af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40cb60 0x55692b4065b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 --2- 172.21.15.96:0/2384588785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b40c170 0x55692b406070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- 172.21.15.96:0/2384588785 >> 172.21.15.96:0/2384588785 conn(0x55692b4076e0 msgr2=0x55692b416690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- 172.21.15.96:0/2384588785 shutdown_connections 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- 172.21.15.96:0/2384588785 wait complete. 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 -- start start 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.169+0000 7f1444e1c740 1 Processor -- start 2024-03-18T15:44:29.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40cb60 0x55692b483a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b411cd0 0x55692b48aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1aa330 con 0x55692b40c170 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1a9c60 con 0x55692b40cb60 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1a6de0 con 0x55692b411cd0 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 10 client.0 _ll_register_callbacks cb 0x7f14340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38146/0 (socket says 172.21.15.96:38146) 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 -- 172.21.15.96:0/1843631870 learned_addr learned my addr 172.21.15.96:0/1843631870 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40cb60 0x55692b483a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b411cd0 0x55692b48aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 -- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b411cd0 msgr2=0x55692b48aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b411cd0 0x55692b48aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 -- 172.21.15.96:0/1843631870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40cb60 msgr2=0x55692b483a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40cb60 0x55692b483a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 -- 172.21.15.96:0/1843631870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692b3389d0 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f143affd640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f143002c710 tx=0x7f142c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c23f6238959af47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f141affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430055070 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1430004780 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142c031070 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1430064050 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 10 client.16659 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.170+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55692b48c1f0 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c031070 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f14300035f0 con 0x55692b40c170 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 10 client.16659 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:29.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 20 client.16659 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b48cb40 0x55692b48ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 4 client.16659 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55692b378d90 con 0x55692b48cb40 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b48cb40 0x55692b48ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f143b7fe640 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b48cb40 0x55692b48ef30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995c1362728b6ad3 server_cookie=ee0995cc36707745 in_seq=0 out_seq=0 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.171+0000 7f141affd640 10 client.16659 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f141affd640 1 -- 172.21.15.96:0/1843631870 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55692b378d90 con 0x55692b48cb40 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f141affd640 10 client.16659 handle_command_reply: tid=0 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 client.16659 shutdown 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 msgr2=0x55692b4834d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f143002c710 tx=0x7f142c0042d0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 shutdown_connections 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55692b411cd0 0x55692b48aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55692b40cb60 0x55692b483a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55692b48cb40 0x55692b48ef30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 --2- 172.21.15.96:0/1843631870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55692b40c170 0x55692b4834d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 >> 172.21.15.96:0/1843631870 conn(0x55692b4076e0 msgr2=0x55692b40a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f141affd640 0 client.16659 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 shutdown_connections 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 1 -- 172.21.15.96:0/1843631870 wait complete. 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1444e1c740 20 client.16659 trim_cache size 0 max 16384 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "201eed03-ac93-4b1d-816a-172ace6d3466": { 2024-03-18T15:44:29.181 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:44:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "201eed03-ac93-4b1d-816a-172ace6d3466", 2024-03-18T15:44:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:44:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:44:29.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:44:29.182 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 msgr2=0x7f14240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 0x7f14240a4cc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f143409ba70 tx=0x7f143406d760 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.172+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 msgr2=0x7f143c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c19eea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1434009fd0 tx=0x7f143400ef10 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 shutdown_connections 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f14240a2800 0x7f14240a4cc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f143c102290 0x7f143c19eea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f143c1018d0 0x7f143c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 --2- 172.21.15.96:0/2778416047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f143c1006e0 0x7f143c0694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 >> 172.21.15.96:0/2778416047 conn(0x7f143c0fbc70 msgr2=0x7f143c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 shutdown_connections 2024-03-18T15:44:29.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:29.173+0000 7f1442234640 1 -- 172.21.15.96:0/2778416047 wait complete. 2024-03-18T15:44:29.184 INFO:tasks.cephfs.filesystem:scrub status for tag:201eed03-ac93-4b1d-816a-172ace6d3466 - {'path': '/', 'tag': '201eed03-ac93-4b1d-816a-172ace6d3466', 'options': 'recursive,force'} 2024-03-18T15:44:29.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:29 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:44:29.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:29 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:44:30.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:29 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:44:30.068 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:29 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:44:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:29 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:44:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:29 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:44:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:44:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:30 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:44:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:30 smithi096 ceph-mon[31930]: pgmap v484: 97 pgs: 97 active+clean; 568 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 461 op/s 2024-03-18T15:44:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:44:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:30 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:44:31.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:30 smithi137 ceph-mon[35868]: pgmap v484: 97 pgs: 97 active+clean; 568 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 461 op/s 2024-03-18T15:44:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:44:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:30 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:44:31.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:30 smithi193 ceph-mon[36702]: pgmap v484: 97 pgs: 97 active+clean; 568 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 461 op/s 2024-03-18T15:44:31.331 INFO:tasks.workunit.client.0.smithi096.stdout: 107 1305264 343 962827 286 733480 1194 2024-03-18T15:44:31.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:31 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:44:32.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:31 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:44:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:31 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:44:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:32 smithi096 ceph-mon[31930]: pgmap v485: 97 pgs: 97 active+clean; 569 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-18T15:44:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:44:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:44:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:44:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:32 smithi137 ceph-mon[35868]: pgmap v485: 97 pgs: 97 active+clean; 569 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-18T15:44:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:44:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:44:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:44:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:32 smithi193 ceph-mon[36702]: pgmap v485: 97 pgs: 97 active+clean; 569 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 283 op/s 2024-03-18T15:44:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:44:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:44:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:44:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:33 smithi137 ceph-mon[35868]: pgmap v486: 97 pgs: 97 active+clean; 567 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-03-18T15:44:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:33 smithi193 ceph-mon[36702]: pgmap v486: 97 pgs: 97 active+clean; 567 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-03-18T15:44:34.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:33 smithi096 ceph-mon[31930]: pgmap v486: 97 pgs: 97 active+clean; 567 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-03-18T15:44:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:36 smithi137 ceph-mon[35868]: pgmap v487: 97 pgs: 97 active+clean; 571 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 316 op/s 2024-03-18T15:44:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:36 smithi193 ceph-mon[36702]: pgmap v487: 97 pgs: 97 active+clean; 571 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 316 op/s 2024-03-18T15:44:36.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:36 smithi096 ceph-mon[31930]: pgmap v487: 97 pgs: 97 active+clean; 571 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 316 op/s 2024-03-18T15:44:38.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:38 smithi137 ceph-mon[35868]: pgmap v488: 97 pgs: 97 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-03-18T15:44:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:38 smithi193 ceph-mon[36702]: pgmap v488: 97 pgs: 97 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-03-18T15:44:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:38 smithi096 ceph-mon[31930]: pgmap v488: 97 pgs: 97 active+clean; 573 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-03-18T15:44:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:40 smithi193 ceph-mon[36702]: pgmap v489: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 507 op/s 2024-03-18T15:44:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:40 smithi096 ceph-mon[31930]: pgmap v489: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 507 op/s 2024-03-18T15:44:41.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:40 smithi137 ceph-mon[35868]: pgmap v489: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 507 op/s 2024-03-18T15:44:41.331 INFO:tasks.workunit.client.0.smithi096.stdout: 111 1556166 292 1166210 257 896162 858 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 -- 172.21.15.96:0/4252095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 msgr2=0x7f39041006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 --2- 172.21.15.96:0/4252095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f39041006e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009ca0 tx=0x7f38fc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 -- 172.21.15.96:0/4252095009 shutdown_connections 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 --2- 172.21.15.96:0/4252095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f39041006e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 --2- 172.21.15.96:0/4252095009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f39040ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 --2- 172.21.15.96:0/4252095009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 -- 172.21.15.96:0/4252095009 >> 172.21.15.96:0/4252095009 conn(0x7f39040fb430 msgr2=0x7f39040fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 -- 172.21.15.96:0/4252095009 shutdown_connections 2024-03-18T15:44:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.334+0000 7f3908ae8640 1 -- 172.21.15.96:0/4252095009 wait complete. 2024-03-18T15:44:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 Processor -- start 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 -- start start 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f390410f690 con 0x7f3904100230 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f390410f510 con 0x7f39040ff870 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3908ae8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f390410f810 con 0x7f3904101d80 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50536/0 (socket says 172.21.15.96:50536) 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33440/0 (socket says 172.21.15.96:33440) 2024-03-18T15:44:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3901d9b640 1 -- 172.21.15.96:0/2554759423 learned_addr learned my addr 172.21.15.96:0/2554759423 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3902d9d640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 -- 172.21.15.96:0/2554759423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 msgr2=0x7f39041a11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041a11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 -- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 msgr2=0x7f390419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f390259c640 1 -- 172.21.15.96:0/2554759423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041a1810 con 0x7f39040ff870 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3902d9d640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.335+0000 7f3901d9b640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f390259c640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38f400e9f0 tx=0x7f38f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f38f37fe640 1 -- 172.21.15.96:0/2554759423 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f400ccc0 con 0x7f39040ff870 2024-03-18T15:44:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f38f37fe640 1 -- 172.21.15.96:0/2554759423 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f4005020 con 0x7f39040ff870 2024-03-18T15:44:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041a1b00 con 0x7f39040ff870 2024-03-18T15:44:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f38f37fe640 1 -- 172.21.15.96:0/2554759423 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f40049e0 con 0x7f39040ff870 2024-03-18T15:44:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.336+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041a2000 con 0x7f39040ff870 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.338+0000 7f390b7a9740 1 Processor -- start 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 -- start start 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a2d30 0x556abe1a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556abe107a40 con 0x556abe1a7ea0 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556abe0b0920 con 0x556abe1a2340 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390b7a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556abe103ae0 con 0x556abe1a2d30 2024-03-18T15:44:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f38f37fe640 1 -- 172.21.15.96:0/2554759423 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f38f4005560 con 0x7f39040ff870 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50542/0 (socket says 172.21.15.96:50542) 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3902d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 -- 172.21.15.96:0/1104149266 learned_addr learned my addr 172.21.15.96:0/1104149266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f390259c640 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a2d30 0x556abe1a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 -- 172.21.15.96:0/1104149266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a2d30 msgr2=0x556abe1a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a2d30 0x556abe1a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 -- 172.21.15.96:0/1104149266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 msgr2=0x556abe1a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1a2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.339+0000 7f3901d9b640 1 -- 172.21.15.96:0/1104149266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abe0c8460 con 0x556abe1a7ea0 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f3902d9d640 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f3901d9b640 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011570 tx=0x7f38f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2fa48446f61cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f38f0ff9640 1 -- 172.21.15.96:0/1104149266 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8005020 con 0x556abe1a7ea0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f38f37fe640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 0x7f38ec0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f38f0ff9640 1 -- 172.21.15.96:0/1104149266 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80047a0 con 0x556abe1a7ea0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f38f0ff9640 1 -- 172.21.15.96:0/1104149266 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800fa30 con 0x556abe1a7ea0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f3902d9d640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 0x7f38ec0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.340+0000 7f38f37fe640 1 -- 172.21.15.96:0/2554759423 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f38f40cea40 con 0x7f39040ff870 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- 172.21.15.96:0/1104149266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 msgr2=0x556abe1aa330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f3902d9d640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 0x7f38ec0a4d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005e30 tx=0x7f38fc032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011570 tx=0x7f38f8011a30 comp rx=0 tx=0).stop 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- 172.21.15.96:0/1104149266 shutdown_connections 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a7ea0 0x556abe1aa330 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a2d30 0x556abe1a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 --2- 172.21.15.96:0/1104149266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- 172.21.15.96:0/1104149266 >> 172.21.15.96:0/1104149266 conn(0x556abe19d8b0 msgr2=0x556abe19dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- 172.21.15.96:0/1104149266 shutdown_connections 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- 172.21.15.96:0/1104149266 wait complete. 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 -- start start 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.341+0000 7f390b7a9740 1 Processor -- start 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556abe1aae50 con 0x556abe1a2d30 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556abe0c3d70 con 0x556abe1a2340 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556abe1aafd0 con 0x556abe1a7ea0 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390b7a9740 10 client.0 _ll_register_callbacks cb 0x556abde1ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50556/0 (socket says 172.21.15.96:50556) 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f3901d9b640 1 -- 172.21.15.96:0/807730454 learned_addr learned my addr 172.21.15.96:0/807730454 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f3902d9d640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 -- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 msgr2=0x556abe1b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 -- 172.21.15.96:0/807730454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 msgr2=0x556abe1acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 -- 172.21.15.96:0/807730454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abde1c210 con 0x556abe1a2340 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f3901d9b640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f390259c640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1ac640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38f40cc560 tx=0x7f38f409b240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf2c1530d1c28f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.342+0000 7f38e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f401c070 con 0x556abe1a2340 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f38f40a36c0 con 0x556abe1a2340 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f80047a0 con 0x556abe1a2340 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f38f409bab0 con 0x556abe1a2340 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 10 client.26474 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556abdee8620 con 0x556abe1a2340 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f80047a0 con 0x556abe1a2340 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f38f4004bc0 con 0x556abe1a2340 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 10 client.26474 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 20 client.26474 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe1b8440 0x556abe1ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 4 client.26474 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556abe114f80 con 0x556abe1b8440 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.343+0000 7f3901d9b640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe1b8440 0x556abe1ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.344+0000 7f38e9ffb640 10 client.26474 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.344+0000 7f3901d9b640 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe1b8440 0x556abe1ba850 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a0dc5d3946fb40 server_cookie=4ce1e36750ab07e1 in_seq=0 out_seq=0 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f38e9ffb640 1 -- 172.21.15.96:0/807730454 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556abe114f80 con 0x556abe1b8440 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f38e9ffb640 10 client.26474 handle_command_reply: tid=0 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 client.26474 shutdown 2024-03-18T15:44:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 msgr2=0x556abe1ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1ac640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38f40cc560 tx=0x7f38f409b240 comp rx=0 tx=0).stop 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 shutdown_connections 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe1b8440 0x556abe1ba850 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2d30 0x556abe1acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f38e9ffb640 0 client.26474 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 --2- 172.21.15.96:0/807730454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2340 0x556abe1ac640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 >> 172.21.15.96:0/807730454 conn(0x556abe19d8b0 msgr2=0x556abe19cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 shutdown_connections 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 1 -- 172.21.15.96:0/807730454 wait complete. 2024-03-18T15:44:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.345+0000 7f390b7a9740 20 client.26474 trim_cache size 0 max 16384 2024-03-18T15:44:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.539+0000 7f390b7a9740 1 Processor -- start 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 -- start start 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe19dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39041aaf30 con 0x556abe1a2340 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39041acb30 con 0x556abe1a2d30 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390b7a9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ab600 con 0x556abe1a7ea0 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33470/0 (socket says 172.21.15.96:33470) 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f3901d9b640 1 -- 172.21.15.96:0/923847094 learned_addr learned my addr 172.21.15.96:0/923847094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:41.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f3902d9d640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe19dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.556+0000 7f390259c640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 -- 172.21.15.96:0/923847094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 msgr2=0x556abe19dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe19dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 -- 172.21.15.96:0/923847094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 msgr2=0x556abe1aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 -- 172.21.15.96:0/923847094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556abe199310 con 0x556abe1a2d30 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f390259c640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:44:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.557+0000 7f3901d9b640 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38f80098c0 tx=0x7f38f8011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614de5aafd0c907c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.560+0000 7f38ebfff640 1 -- 172.21.15.96:0/923847094 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8004d80 con 0x556abe1a2d30 2024-03-18T15:44:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.560+0000 7f38ebfff640 1 -- 172.21.15.96:0/923847094 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8005170 con 0x556abe1a2d30 2024-03-18T15:44:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.560+0000 7f38ebfff640 1 -- 172.21.15.96:0/923847094 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8029340 con 0x556abe1a2d30 2024-03-18T15:44:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.563+0000 7f390b7a9740 1 -- 172.21.15.96:0/923847094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 msgr2=0x556abe1b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.563+0000 7f390b7a9740 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38f80098c0 tx=0x7f38f8011a30 comp rx=0 tx=0).stop 2024-03-18T15:44:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.578+0000 7f390b7a9740 1 -- 172.21.15.96:0/923847094 shutdown_connections 2024-03-18T15:44:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.578+0000 7f390b7a9740 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe19dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.578+0000 7f390b7a9740 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b3b80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.578+0000 7f390b7a9740 1 --2- 172.21.15.96:0/923847094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.578+0000 7f390b7a9740 1 -- 172.21.15.96:0/923847094 >> 172.21.15.96:0/923847094 conn(0x556abe19d8b0 msgr2=0x556abe1a7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.584+0000 7f390b7a9740 1 -- 172.21.15.96:0/923847094 shutdown_connections 2024-03-18T15:44:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.584+0000 7f390b7a9740 1 -- 172.21.15.96:0/923847094 wait complete. 2024-03-18T15:44:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.585+0000 7f390b7a9740 1 -- start start 2024-03-18T15:44:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.585+0000 7f390b7a9740 1 Processor -- start 2024-03-18T15:44:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f390b7a9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f3901d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33474/0 (socket says 172.21.15.96:33474) 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f3901d9b640 1 -- 172.21.15.96:0/3312151303 learned_addr learned my addr 172.21.15.96:0/3312151303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f3902d9d640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.587+0000 7f390259c640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.586+0000 7f390b7a9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39041ac460 con 0x556abe1a2340 2024-03-18T15:44:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f39041abd90 con 0x556abe1a2d30 2024-03-18T15:44:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39041a8f10 con 0x556abe1a7ea0 2024-03-18T15:44:41.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f390b7a9740 10 client.0 _ll_register_callbacks cb 0x7f38f400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 -- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 msgr2=0x556abe1b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 -- 172.21.15.96:0/3312151303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 msgr2=0x556abe1b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556abe1b2ab0 con 0x556abe1a2d30 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f390259c640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:44:41.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.588+0000 7f3901d9b640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011510 tx=0x7f38f8004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd06ebf12ed1493 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.601+0000 7f38e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.601+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8037020 con 0x556abe1a2d30 2024-03-18T15:44:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.601+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38f8005170 con 0x556abe1a2d30 2024-03-18T15:44:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.601+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f38f8016040 con 0x556abe1a2d30 2024-03-18T15:44:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.602+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f38f8005530 con 0x556abe1a2d30 2024-03-18T15:44:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.602+0000 7f390b7a9740 10 client.26480 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.602+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556abe1b3da0 con 0x556abe1a2d30 2024-03-18T15:44:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8005170 con 0x556abe1a2d30 2024-03-18T15:44:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f38f80035f0 con 0x556abe1a2d30 2024-03-18T15:44:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 10 client.26480 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 20 client.26480 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe21ecf0 0x556abe221110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 4 client.26480 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556abe10f2d0 con 0x556abe21ecf0 2024-03-18T15:44:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.604+0000 7f390259c640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe21ecf0 0x556abe221110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.605+0000 7f38e9ffb640 10 client.26480 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:41.605+0000 7f390259c640 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe21ecf0 0x556abe221110 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de11a9c33b0db93 server_cookie=ac6e3c1640a2724 in_seq=0 out_seq=0 2024-03-18T15:44:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f38e9ffb640 1 -- 172.21.15.96:0/3312151303 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556abe10f2d0 con 0x556abe21ecf0 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f38e9ffb640 10 client.26480 handle_command_reply: tid=0 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 client.26480 shutdown 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 msgr2=0x556abe1b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38f8011510 tx=0x7f38f8004d80 comp rx=0 tx=0).stop 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 shutdown_connections 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556abe1a7ea0 0x556abe1b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556abe1a2d30 0x556abe1b14b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556abe21ecf0 0x556abe221110 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 --2- 172.21.15.96:0/3312151303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556abe1a2340 0x556abe1b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f38e9ffb640 0 client.26480 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 >> 172.21.15.96:0/3312151303 conn(0x556abe19d8b0 msgr2=0x556abe1a3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 shutdown_connections 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 1 -- 172.21.15.96:0/3312151303 wait complete. 2024-03-18T15:44:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.385+0000 7f390b7a9740 20 client.26480 trim_cache size 0 max 16384 2024-03-18T15:44:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:44:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:44:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:44:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 msgr2=0x7f38ec0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 0x7f38ec0a4d60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005e30 tx=0x7f38fc032590 comp rx=0 tx=0).stop 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 msgr2=0x7f3904199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f38f400e9f0 tx=0x7f38f400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 shutdown_connections 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f38ec0a28a0 0x7f38ec0a4d60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3904101d80 0x7f39041a11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3904100230 0x7f390419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 --2- 172.21.15.96:0/2554759423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f39040ff870 0x7f3904199bf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 >> 172.21.15.96:0/2554759423 conn(0x7f39040fb430 msgr2=0x7f39040fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 shutdown_connections 2024-03-18T15:44:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:42.386+0000 7f3908ae8640 1 -- 172.21.15.96:0/2554759423 wait complete. 2024-03-18T15:44:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:42 smithi193 ceph-mon[36702]: pgmap v490: 97 pgs: 97 active+clean; 609 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 521 op/s 2024-03-18T15:44:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:42 smithi096 ceph-mon[31930]: pgmap v490: 97 pgs: 97 active+clean; 609 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 521 op/s 2024-03-18T15:44:43.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:42 smithi137 ceph-mon[35868]: pgmap v490: 97 pgs: 97 active+clean; 609 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 521 op/s 2024-03-18T15:44:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:44 smithi193 ceph-mon[36702]: pgmap v491: 97 pgs: 97 active+clean; 618 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 623 op/s 2024-03-18T15:44:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:44 smithi096 ceph-mon[31930]: pgmap v491: 97 pgs: 97 active+clean; 618 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 623 op/s 2024-03-18T15:44:45.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:44 smithi137 ceph-mon[35868]: pgmap v491: 97 pgs: 97 active+clean; 618 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 623 op/s 2024-03-18T15:44:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:46 smithi193 ceph-mon[36702]: pgmap v492: 97 pgs: 97 active+clean; 620 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-18T15:44:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:46 smithi096 ceph-mon[31930]: pgmap v492: 97 pgs: 97 active+clean; 620 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-18T15:44:47.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:46 smithi137 ceph-mon[35868]: pgmap v492: 97 pgs: 97 active+clean; 620 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.2 MiB/s wr, 610 op/s 2024-03-18T15:44:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:48 smithi193 ceph-mon[36702]: pgmap v493: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 569 op/s 2024-03-18T15:44:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:48 smithi096 ceph-mon[31930]: pgmap v493: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 569 op/s 2024-03-18T15:44:49.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:48 smithi137 ceph-mon[35868]: pgmap v493: 97 pgs: 97 active+clean; 619 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 569 op/s 2024-03-18T15:44:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:50 smithi193 ceph-mon[36702]: pgmap v494: 97 pgs: 97 active+clean; 600 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 633 op/s 2024-03-18T15:44:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:50 smithi096 ceph-mon[31930]: pgmap v494: 97 pgs: 97 active+clean; 600 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 633 op/s 2024-03-18T15:44:51.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:50 smithi137 ceph-mon[35868]: pgmap v494: 97 pgs: 97 active+clean; 600 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 633 op/s 2024-03-18T15:44:51.331 INFO:tasks.workunit.client.0.smithi096.stdout: 119 1942847 467 1449325 273 1141693 1170 2024-03-18T15:44:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:52 smithi193 ceph-mon[36702]: pgmap v495: 97 pgs: 97 active+clean; 599 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-03-18T15:44:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:52 smithi096 ceph-mon[31930]: pgmap v495: 97 pgs: 97 active+clean; 599 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-03-18T15:44:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:52 smithi137 ceph-mon[35868]: pgmap v495: 97 pgs: 97 active+clean; 599 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 455 op/s 2024-03-18T15:44:54.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:54 smithi137 ceph-mon[35868]: pgmap v496: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-03-18T15:44:54.917 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:54 smithi193 ceph-mon[36702]: pgmap v496: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-03-18T15:44:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:54 smithi096 ceph-mon[31930]: pgmap v496: 97 pgs: 97 active+clean; 596 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 441 op/s 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 -- 172.21.15.96:0/4123909003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 msgr2=0x7f033010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 --2- 172.21.15.96:0/4123909003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f033010e1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ca0 tx=0x7f032802c140 comp rx=0 tx=0).stop 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 -- 172.21.15.96:0/4123909003 shutdown_connections 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 --2- 172.21.15.96:0/4123909003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f033010e1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 --2- 172.21.15.96:0/4123909003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 0x7f0330108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 --2- 172.21.15.96:0/4123909003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.828+0000 7f03360c6640 1 -- 172.21.15.96:0/4123909003 >> 172.21.15.96:0/4123909003 conn(0x7f033006e650 msgr2=0x7f0330070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.829+0000 7f03360c6640 1 -- 172.21.15.96:0/4123909003 shutdown_connections 2024-03-18T15:44:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.829+0000 7f03360c6640 1 -- 172.21.15.96:0/4123909003 wait complete. 2024-03-18T15:44:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.830+0000 7f03360c6640 1 Processor -- start 2024-03-18T15:44:55.831 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 -- start start 2024-03-18T15:44:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 0x7f0330115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.832 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f033011c170 con 0x7f0330072ab0 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f033011bff0 con 0x7f0330073410 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f03360c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f033011c2f0 con 0x7f0330109510 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f032ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f032ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:58240/0 (socket says 172.21.15.96:58240) 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f032ffff640 1 -- 172.21.15.96:0/3539047222 learned_addr learned my addr 172.21.15.96:0/3539047222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f032effd640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 0x7f0330115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.831+0000 7f032f7fe640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032ffff640 1 -- 172.21.15.96:0/3539047222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 msgr2=0x7f0330115410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032ffff640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 0x7f0330115410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032ffff640 1 -- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 msgr2=0x7f0330114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032ffff640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032ffff640 1 -- 172.21.15.96:0/3539047222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03301b6dc0 con 0x7f0330109510 2024-03-18T15:44:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.832+0000 7f032f7fe640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:44:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.833+0000 7f032ffff640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0328037040 tx=0x7f0328035d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.848+0000 7f032cff9640 1 -- 172.21.15.96:0/3539047222 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328039040 con 0x7f0330109510 2024-03-18T15:44:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.849+0000 7f032cff9640 1 -- 172.21.15.96:0/3539047222 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0328044070 con 0x7f0330109510 2024-03-18T15:44:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.849+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301b6f90 con 0x7f0330109510 2024-03-18T15:44:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.849+0000 7f032cff9640 1 -- 172.21.15.96:0/3539047222 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328004310 con 0x7f0330109510 2024-03-18T15:44:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.849+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301b73c0 con 0x7f0330109510 2024-03-18T15:44:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.853+0000 7f032cff9640 1 -- 172.21.15.96:0/3539047222 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0328003a00 con 0x7f0330109510 2024-03-18T15:44:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.858+0000 7f0338d92740 1 Processor -- start 2024-03-18T15:44:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 -- start start 2024-03-18T15:44:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 0x563ca60b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ca601ca40 con 0x563ca60b7500 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ca5fc5920 con 0x563ca60bd060 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f0338d92740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ca6018ae0 con 0x563ca60b7ef0 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:58256/0 (socket says 172.21.15.96:58256) 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.859+0000 7f032f7fe640 1 -- 172.21.15.96:0/1630983670 learned_addr learned my addr 172.21.15.96:0/1630983670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032f7fe640 1 -- 172.21.15.96:0/1630983670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 msgr2=0x563ca60bf4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032f7fe640 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60bf4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032f7fe640 1 -- 172.21.15.96:0/1630983670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 msgr2=0x563ca60b7920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032effd640 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 0x563ca60b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032cff9640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 0x7f03180a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032f7fe640 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 0x563ca60b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032f7fe640 1 -- 172.21.15.96:0/1630983670 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca5fdd460 con 0x563ca60b7ef0 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032effd640 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 0x563ca60b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.860+0000 7f032cff9640 1 -- 172.21.15.96:0/3539047222 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f03280f1050 con 0x7f0330109510 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.861+0000 7f032effd640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 0x7f03180a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.861+0000 7f032f7fe640 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c60 tx=0x7f0320010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c80a2e690a7e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.862+0000 7f032effd640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 0x7f03180a4ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0324006a70 tx=0x7f03240093b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.867+0000 7f0315ffb640 1 -- 172.21.15.96:0/1630983670 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320005020 con 0x563ca60b7ef0 2024-03-18T15:44:55.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.871+0000 7f0315ffb640 1 -- 172.21.15.96:0/1630983670 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03200047a0 con 0x563ca60b7ef0 2024-03-18T15:44:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 -- 172.21.15.96:0/1630983670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 msgr2=0x563ca60bc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c60 tx=0x7f0320010740 comp rx=0 tx=0).stop 2024-03-18T15:44:55.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 -- 172.21.15.96:0/1630983670 shutdown_connections 2024-03-18T15:44:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7ef0 0x563ca60bc9f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 --2- 172.21.15.96:0/1630983670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7500 0x563ca60b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 -- 172.21.15.96:0/1630983670 >> 172.21.15.96:0/1630983670 conn(0x563ca60b2a70 msgr2=0x563ca60b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:55.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 -- 172.21.15.96:0/1630983670 shutdown_connections 2024-03-18T15:44:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.872+0000 7f0338d92740 1 -- 172.21.15.96:0/1630983670 wait complete. 2024-03-18T15:44:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.873+0000 7f0338d92740 1 -- start start 2024-03-18T15:44:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.873+0000 7f0338d92740 1 Processor -- start 2024-03-18T15:44:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563ca60c0060 con 0x563ca60b7ef0 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ca5fd8d70 con 0x563ca60bd060 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ca60c01e0 con 0x563ca60b7500 2024-03-18T15:44:55.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f0338d92740 10 client.0 _ll_register_callbacks cb 0x563ca5d30a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38484/0 (socket says 172.21.15.96:38484) 2024-03-18T15:44:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 -- 172.21.15.96:0/2048650246 learned_addr learned my addr 172.21.15.96:0/2048650246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 -- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 msgr2=0x563ca60c1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 -- 172.21.15.96:0/2048650246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 msgr2=0x563ca60c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032ffff640 1 -- 172.21.15.96:0/2048650246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca5d31210 con 0x563ca60bd060 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032f7fe640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.874+0000 7f032effd640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:44:55.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.875+0000 7f032ffff640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03280c5bd0 tx=0x7f03280bd560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38837247197a1aca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.887+0000 7f030effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.887+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280bd960 con 0x563ca60bd060 2024-03-18T15:44:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.887+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03200047a0 con 0x563ca60bd060 2024-03-18T15:44:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.887+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f03280b9fd0 con 0x563ca60bd060 2024-03-18T15:44:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.888+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f03280c1ab0 con 0x563ca60bd060 2024-03-18T15:44:55.889 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 10 client.26486 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ca5dfd620 con 0x563ca60bd060 2024-03-18T15:44:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03200047a0 con 0x563ca60bd060 2024-03-18T15:44:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f032803c030 con 0x563ca60bd060 2024-03-18T15:44:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 10 client.26486 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 20 client.26486 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60cd790 0x563ca60cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 4 client.26486 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.889+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca6029f80 con 0x563ca60cd790 2024-03-18T15:44:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.890+0000 7f032f7fe640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60cd790 0x563ca60cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.890+0000 7f032f7fe640 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60cd790 0x563ca60cfbe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6f9a35f22e0092 server_cookie=4baf2f1705bdb5e1 in_seq=0 out_seq=0 2024-03-18T15:44:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.890+0000 7f030effd640 10 client.26486 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.892+0000 7f030effd640 1 -- 172.21.15.96:0/2048650246 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563ca6029f80 con 0x563ca60cd790 2024-03-18T15:44:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.892+0000 7f030effd640 10 client.26486 handle_command_reply: tid=0 2024-03-18T15:44:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.892+0000 7f0338d92740 1 client.26486 shutdown 2024-03-18T15:44:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.902+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 msgr2=0x563ca60c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.902+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03280c5bd0 tx=0x7f03280bd560 comp rx=0 tx=0).stop 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 shutdown_connections 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60bd060 0x563ca60c8dc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60cd790 0x563ca60cfbe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 --2- 172.21.15.96:0/2048650246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.903+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 >> 172.21.15.96:0/2048650246 conn(0x563ca60b2a70 msgr2=0x563ca60b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.926+0000 7f030effd640 0 client.26486 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.927+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 shutdown_connections 2024-03-18T15:44:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.927+0000 7f0338d92740 1 -- 172.21.15.96:0/2048650246 wait complete. 2024-03-18T15:44:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.927+0000 7f0338d92740 20 client.26486 trim_cache size 0 max 16384 2024-03-18T15:44:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.988+0000 7f0338d92740 1 Processor -- start 2024-03-18T15:44:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.988+0000 7f0338d92740 1 -- start start 2024-03-18T15:44:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.989+0000 7f0338d92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.989+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.989+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43340/0 (socket says 172.21.15.96:43340) 2024-03-18T15:44:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f032f7fe640 1 -- 172.21.15.96:0/1846157871 learned_addr learned my addr 172.21.15.96:0/1846157871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 0x563ca60c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c02b0 con 0x563ca60b2a70 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f032effd640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f032ffff640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 0x563ca60c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c1eb0 con 0x563ca60b7ef0 2024-03-18T15:44:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.990+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c0980 con 0x563ca60b7500 2024-03-18T15:44:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 -- 172.21.15.96:0/1846157871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 msgr2=0x563ca60b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 -- 172.21.15.96:0/1846157871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 msgr2=0x563ca60b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 -- 172.21.15.96:0/1846157871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca60b37e0 con 0x563ca60b7ef0 2024-03-18T15:44:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032f7fe640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:55.991+0000 7f032ffff640 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 0x563ca60c8d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03280ba330 tx=0x7f03280bbe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30125fffcc70749c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.003+0000 7f03157fa640 1 -- 172.21.15.96:0/1846157871 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280bd960 con 0x563ca60b7ef0 2024-03-18T15:44:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.003+0000 7f03157fa640 1 -- 172.21.15.96:0/1846157871 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03280c6100 con 0x563ca60b7ef0 2024-03-18T15:44:56.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.003+0000 7f03157fa640 1 -- 172.21.15.96:0/1846157871 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328004b40 con 0x563ca60b7ef0 2024-03-18T15:44:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.006+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 msgr2=0x563ca60c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.006+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 0x563ca60c8d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f03280ba330 tx=0x7f03280bbe70 comp rx=0 tx=0).stop 2024-03-18T15:44:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 shutdown_connections 2024-03-18T15:44:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b7ef0 0x563ca60c8d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 --2- 172.21.15.96:0/1846157871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b2a70 0x563ca60b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 >> 172.21.15.96:0/1846157871 conn(0x563ca60bcb30 msgr2=0x563ca60c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 shutdown_connections 2024-03-18T15:44:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.010+0000 7f0338d92740 1 -- 172.21.15.96:0/1846157871 wait complete. 2024-03-18T15:44:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.011+0000 7f0338d92740 1 -- start start 2024-03-18T15:44:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.011+0000 7f0338d92740 1 Processor -- start 2024-03-18T15:44:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c17e0 con 0x563ca60b7ef0 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03301c1110 con 0x563ca60b2a70 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f03301be290 con 0x563ca60b7500 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f0338d92740 10 client.0 _ll_register_callbacks cb 0x7f03280b9cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38506/0 (socket says 172.21.15.96:38506) 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 -- 172.21.15.96:0/3544667305 learned_addr learned my addr 172.21.15.96:0/3544667305 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032effd640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032ffff640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 -- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 msgr2=0x563ca60c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 -- 172.21.15.96:0/3544667305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 msgr2=0x563ca60c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.012+0000 7f032f7fe640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.013+0000 7f032f7fe640 1 -- 172.21.15.96:0/3544667305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca60c7d00 con 0x563ca60b2a70 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.013+0000 7f032ffff640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:44:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.013+0000 7f032f7fe640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c60 tx=0x7f0320009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb854d000b15003 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.023+0000 7f030effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.024+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032002f060 con 0x563ca60b2a70 2024-03-18T15:44:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.024+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03280c6100 con 0x563ca60b2a70 2024-03-18T15:44:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.024+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f0320012a30 con 0x563ca60b2a70 2024-03-18T15:44:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.027+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0320012c30 con 0x563ca60b2a70 2024-03-18T15:44:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.028+0000 7f0338d92740 10 client.26492 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.028+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563ca60c9010 con 0x563ca60b2a70 2024-03-18T15:44:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.029+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03280c6100 con 0x563ca60b2a70 2024-03-18T15:44:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.029+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f032000f040 con 0x563ca60b2a70 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 10 client.26492 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 20 client.26492 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60bd060 0x563ca6135e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 4 client.26492 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ca60242d0 con 0x563ca60bd060 2024-03-18T15:44:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.030+0000 7f032effd640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60bd060 0x563ca6135e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.031+0000 7f030effd640 10 client.26492 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.031+0000 7f032effd640 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60bd060 0x563ca6135e70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba8728f1153abba server_cookie=df32b62677e1d010 in_seq=0 out_seq=0 2024-03-18T15:44:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.042+0000 7f030effd640 1 -- 172.21.15.96:0/3544667305 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ca60242d0 con 0x563ca60bd060 2024-03-18T15:44:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.042+0000 7f030effd640 10 client.26492 handle_command_reply: tid=0 2024-03-18T15:44:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.046+0000 7f0338d92740 1 client.26492 shutdown 2024-03-18T15:44:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 msgr2=0x563ca60c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0320009c60 tx=0x7f0320009f80 comp rx=0 tx=0).stop 2024-03-18T15:44:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 shutdown_connections 2024-03-18T15:44:56.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563ca60bd060 0x563ca6135e70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563ca60b7ef0 0x563ca60c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca60b7500 0x563ca60c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 --2- 172.21.15.96:0/3544667305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ca60b2a70 0x563ca60c6210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 >> 172.21.15.96:0/3544667305 conn(0x563ca60bcb30 msgr2=0x563ca60c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f030effd640 0 client.26492 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 shutdown_connections 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.053+0000 7f0338d92740 1 -- 172.21.15.96:0/3544667305 wait complete. 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.054+0000 7f0338d92740 20 client.26492 trim_cache size 0 max 16384 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:44:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:44:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:44:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:44:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.072+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 msgr2=0x7f03180a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.072+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 0x7f03180a4ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0324006a70 tx=0x7f03240093b0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.072+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 msgr2=0x7f03301b6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.072+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0328037040 tx=0x7f0328035d10 comp rx=0 tx=0).stop 2024-03-18T15:44:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.075+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 shutdown_connections 2024-03-18T15:44:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.075+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f03180a2a00 0x7f03180a4ec0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.075+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0330109510 0x7f03301b6880 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.075+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0330073410 0x7f0330115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.075+0000 7f03360c6640 1 --2- 172.21.15.96:0/3539047222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0330072ab0 0x7f0330114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.076+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 >> 172.21.15.96:0/3539047222 conn(0x7f033006e650 msgr2=0x7f03300702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.077+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 shutdown_connections 2024-03-18T15:44:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:44:56.077+0000 7f03360c6640 1 -- 172.21.15.96:0/3539047222 wait complete. 2024-03-18T15:44:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:56 smithi193 ceph-mon[36702]: pgmap v497: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-03-18T15:44:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:56 smithi096 ceph-mon[31930]: pgmap v497: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-03-18T15:44:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:56 smithi137 ceph-mon[35868]: pgmap v497: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-03-18T15:44:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:44:58 smithi193 ceph-mon[36702]: pgmap v498: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-18T15:44:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:44:58 smithi096 ceph-mon[31930]: pgmap v498: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-18T15:44:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:44:58 smithi137 ceph-mon[35868]: pgmap v498: 97 pgs: 97 active+clean; 598 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 285 op/s 2024-03-18T15:44:59.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:44:59.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:44:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 -- 172.21.15.96:0/2451240373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2081018d0 msgr2=0x7fb208101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/2451240373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2081018d0 0x7fb208101d50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 -- 172.21.15.96:0/2451240373 shutdown_connections 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/2451240373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb208102290 0x7fb208108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/2451240373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2081018d0 0x7fb208101d50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/2451240373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 0x7fb208100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 -- 172.21.15.96:0/2451240373 >> 172.21.15.96:0/2451240373 conn(0x7fb2080fbc70 msgr2=0x7fb2080fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 -- 172.21.15.96:0/2451240373 shutdown_connections 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.293+0000 7fb20d8dc640 1 -- 172.21.15.96:0/2451240373 wait complete. 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 Processor -- start 2024-03-18T15:44:59.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 -- start start 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 0x7fb20810b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 0x7fb2081096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb20810d480 con 0x7fb208102290 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb20810d300 con 0x7fb2081018d0 2024-03-18T15:44:59.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb20d8dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb20810d600 con 0x7fb2081006e0 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb2077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb2077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43370/0 (socket says 172.21.15.96:43370) 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb2077fe640 1 -- 172.21.15.96:0/676395343 learned_addr learned my addr 172.21.15.96:0/676395343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb2067fc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 0x7fb2081096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.294+0000 7fb206ffd640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 0x7fb20810b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 -- 172.21.15.96:0/676395343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 msgr2=0x7fb20810b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 0x7fb20810b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 -- 172.21.15.96:0/676395343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 msgr2=0x7fb2081096a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 0x7fb2081096a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 -- 172.21.15.96:0/676395343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20810a470 con 0x7fb208102290 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2067fc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 0x7fb2081096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb2077fe640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb20000eb10 tx=0x7fb20000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb20c8da640 1 -- 172.21.15.96:0/676395343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200004b20 con 0x7fb208102290 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081a1c60 con 0x7fb208102290 2024-03-18T15:44:59.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb20c8da640 1 -- 172.21.15.96:0/676395343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb200004cc0 con 0x7fb208102290 2024-03-18T15:44:59.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb20c8da640 1 -- 172.21.15.96:0/676395343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200010760 con 0x7fb208102290 2024-03-18T15:44:59.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.295+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a2130 con 0x7fb208102290 2024-03-18T15:44:59.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.297+0000 7fb20c8da640 1 -- 172.21.15.96:0/676395343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb200005020 con 0x7fb208102290 2024-03-18T15:44:59.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.297+0000 7fb2105a0740 1 Processor -- start 2024-03-18T15:44:59.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.297+0000 7fb2105a0740 1 -- start start 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0170 0x56554ded0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554ded8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56554de35500 con 0x56554ded5cd0 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56554ddddbe0 con 0x56554ded0b60 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2105a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56554de315a0 con 0x56554ded0170 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb20c8da640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 0x7fb1f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554ded8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0170 0x56554ded0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38530/0 (socket says 172.21.15.96:38530) 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 -- 172.21.15.96:0/384780275 learned_addr learned my addr 172.21.15.96:0/384780275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2077fe640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 0x7fb1f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb20c8da640 1 -- 172.21.15.96:0/676395343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb2000cefd0 con 0x7fb208102290 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 -- 172.21.15.96:0/384780275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0170 msgr2=0x56554ded0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0170 0x56554ded0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 -- 172.21.15.96:0/384780275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 msgr2=0x56554ded8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554ded8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 -- 172.21.15.96:0/384780275 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554ddf14b0 con 0x56554ded0b60 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb206ffd640 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ca0 tx=0x7fb1f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79bc2cd87b598532 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb2077fe640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 0x7fb1f00a4cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb200097e40 tx=0x7fb2000667a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.298+0000 7fb1edffb640 1 -- 172.21.15.96:0/384780275 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8009e50 con 0x56554ded0b60 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb1edffb640 1 -- 172.21.15.96:0/384780275 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f80035f0 con 0x56554ded0b60 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb1edffb640 1 -- 172.21.15.96:0/384780275 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8019630 con 0x56554ded0b60 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- 172.21.15.96:0/384780275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 msgr2=0x56554ded5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009ca0 tx=0x7fb1f8011810 comp rx=0 tx=0).stop 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- 172.21.15.96:0/384780275 shutdown_connections 2024-03-18T15:44:59.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554ded8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554ded5660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 --2- 172.21.15.96:0/384780275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0170 0x56554ded0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- 172.21.15.96:0/384780275 >> 172.21.15.96:0/384780275 conn(0x56554decb6e0 msgr2=0x56554decbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- 172.21.15.96:0/384780275 shutdown_connections 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- 172.21.15.96:0/384780275 wait complete. 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.299+0000 7fb2105a0740 1 -- start start 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 Processor -- start 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554deda690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56554ded8c80 con 0x56554ded5cd0 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56554ddf1160 con 0x56554ded0170 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56554ded8e00 con 0x56554ded0b60 2024-03-18T15:44:59.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2105a0740 10 client.0 _ll_register_callbacks cb 0x56554ddd5150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43380/0 (socket says 172.21.15.96:43380) 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 -- 172.21.15.96:0/4101829426 learned_addr learned my addr 172.21.15.96:0/4101829426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb206ffd640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554deda690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 -- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 msgr2=0x56554dedabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 -- 172.21.15.96:0/4101829426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 msgr2=0x56554deda690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554deda690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.300+0000 7fb2067fc640 1 -- 172.21.15.96:0/4101829426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554dc137d0 con 0x56554ded5cd0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2067fc640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc036040 tx=0x7fb1fc02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb442fae0648adf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc034440 con 0x56554ded5cd0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb1fc0345e0 con 0x56554ded5cd0 2024-03-18T15:44:59.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f80035f0 con 0x56554ded5cd0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc0347e0 con 0x56554ded5cd0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 10 client.16698 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56554db493f0 con 0x56554ded5cd0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f80035f0 con 0x56554ded5cd0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb1fc0419c0 con 0x56554ded5cd0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 10 client.16698 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 20 client.16698 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554dee60b0 0x56554dee8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 4 client.16698 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.301+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554de42a40 con 0x56554dee60b0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.302+0000 7fb206ffd640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554dee60b0 0x56554dee8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.302+0000 7fb206ffd640 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554dee60b0 0x56554dee8550 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad3769196cf76f9 server_cookie=7b36bcc2206bde5b in_seq=0 out_seq=0 2024-03-18T15:44:59.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.302+0000 7fb1e6ffd640 10 client.16698 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/4101829426 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56554de42a40 con 0x56554dee60b0 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb1e6ffd640 10 client.16698 handle_command_reply: tid=0 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 client.16698 shutdown 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 msgr2=0x56554dee1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc036040 tx=0x7fb1fc02c6d0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 shutdown_connections 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554dee60b0 0x56554dee8550 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee1c10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb1e6ffd640 0 client.16698 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 --2- 172.21.15.96:0/4101829426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554deda690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 >> 172.21.15.96:0/4101829426 conn(0x56554decb6e0 msgr2=0x56554deca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 shutdown_connections 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 1 -- 172.21.15.96:0/4101829426 wait complete. 2024-03-18T15:44:59.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.303+0000 7fb2105a0740 20 client.16698 trim_cache size 0 max 16384 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.320+0000 7fb2105a0740 1 Processor -- start 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.320+0000 7fb2105a0740 1 -- start start 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554dee1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded5cd0 0x56554deca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081aaef0 con 0x56554ded0170 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081acaf0 con 0x56554ded0b60 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2105a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ab5c0 con 0x56554ded5cd0 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43382/0 (socket says 172.21.15.96:43382) 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 -- 172.21.15.96:0/3672496962 learned_addr learned my addr 172.21.15.96:0/3672496962 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb206ffd640 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554dee1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded5cd0 0x56554deca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 -- 172.21.15.96:0/3672496962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded5cd0 msgr2=0x56554deca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded5cd0 0x56554deca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 -- 172.21.15.96:0/3672496962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 msgr2=0x56554dee1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554dee1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 -- 172.21.15.96:0/3672496962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554db090f0 con 0x56554ded0170 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb2067fc640 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009850 tx=0x7fb1fc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe0cae5fed23116 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb1ed7fa640 1 -- 172.21.15.96:0/3672496962 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc009520 con 0x56554ded0170 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb1ed7fa640 1 -- 172.21.15.96:0/3672496962 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1fc034440 con 0x56554ded0170 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.321+0000 7fb1ed7fa640 1 -- 172.21.15.96:0/3672496962 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc041970 con 0x56554ded0170 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- 172.21.15.96:0/3672496962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 msgr2=0x56554deca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009850 tx=0x7fb1fc005dd0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- 172.21.15.96:0/3672496962 shutdown_connections 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded5cd0 0x56554deca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0b60 0x56554dee1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 --2- 172.21.15.96:0/3672496962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded0170 0x56554deca070 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- 172.21.15.96:0/3672496962 >> 172.21.15.96:0/3672496962 conn(0x56554decb6e0 msgr2=0x56554decbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- 172.21.15.96:0/3672496962 shutdown_connections 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- 172.21.15.96:0/3672496962 wait complete. 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 -- start start 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.322+0000 7fb2105a0740 1 Processor -- start 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554dedf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ac420 con 0x56554ded5cd0 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081abd50 con 0x56554ded0170 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081a8ed0 con 0x56554ded0b60 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 10 client.0 _ll_register_callbacks cb 0x7fb1fc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb206ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb206ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:58326/0 (socket says 172.21.15.96:58326) 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43384/0 (socket says 172.21.15.96:43384) 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb206ffd640 1 -- 172.21.15.96:0/354392433 learned_addr learned my addr 172.21.15.96:0/354392433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554dedf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 -- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 msgr2=0x56554dedfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 -- 172.21.15.96:0/354392433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 msgr2=0x56554dedf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554dedf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 -- 172.21.15.96:0/354392433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554ddfc9d0 con 0x56554ded5cd0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb206ffd640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2067fc640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc041d60 tx=0x7fb1fc043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657203b2aadb1966 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb1e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc05f060 con 0x56554ded5cd0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fb1fc034d90 con 0x56554ded5cd0 2024-03-18T15:44:59.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.323+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1fc034440 con 0x56554ded5cd0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc009520 con 0x56554ded5cd0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 10 client.16704 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56554df4c720 con 0x56554ded5cd0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1fc034440 con 0x56554ded5cd0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fb1fc05bb20 con 0x56554ded5cd0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 10 client.16704 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 20 client.16704 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554df4cfc0 0x56554df4f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 4 client.16704 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56554de3cd90 con 0x56554df4cfc0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb206ffd640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554df4cfc0 0x56554df4f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb1e6ffd640 10 client.16704 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:59.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.324+0000 7fb206ffd640 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554df4cfc0 0x56554df4f3e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5109bcd768a8c88 server_cookie=66273b329aa3a520 in_seq=0 out_seq=0 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb1e6ffd640 1 -- 172.21.15.96:0/354392433 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56554de3cd90 con 0x56554df4cfc0 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb1e6ffd640 10 client.16704 handle_command_reply: tid=0 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 client.16704 shutdown 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 msgr2=0x56554dee0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc041d60 tx=0x7fb1fc043040 comp rx=0 tx=0).stop 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 shutdown_connections 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56554df4cfc0 0x56554df4f3e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56554ded5cd0 0x56554dee0d00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56554ded0b60 0x56554dedfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 --2- 172.21.15.96:0/354392433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56554ded0170 0x56554dedf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb1e6ffd640 0 client.16704 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 >> 172.21.15.96:0/354392433 conn(0x56554decb6e0 msgr2=0x56554dece940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 shutdown_connections 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 1 -- 172.21.15.96:0/354392433 wait complete. 2024-03-18T15:44:59.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb2105a0740 20 client.16704 trim_cache size 0 max 16384 2024-03-18T15:44:59.333 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:44:59.333 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:44:59.333 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:44:59.333 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 msgr2=0x7fb1f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 0x7fb1f00a4cc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb200097e40 tx=0x7fb2000667a0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 msgr2=0x7fb208109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.325+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb20000eb10 tx=0x7fb20000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 shutdown_connections 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb1f00a2800 0x7fb1f00a4cc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb208102290 0x7fb208109be0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2081018d0 0x7fb2081096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 --2- 172.21.15.96:0/676395343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2081006e0 0x7fb20810b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 >> 172.21.15.96:0/676395343 conn(0x7fb2080fbc70 msgr2=0x7fb208104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 shutdown_connections 2024-03-18T15:44:59.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.326+0000 7fb20d8dc640 1 -- 172.21.15.96:0/676395343 wait complete. 2024-03-18T15:44:59.337 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:44:59.337 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:44:59.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 -- 172.21.15.96:0/811272022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 msgr2=0x7fdc10101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/811272022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10101c20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009900 tx=0x7fdc0c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 -- 172.21.15.96:0/811272022 shutdown_connections 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/811272022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101033b0 0x7fdc10109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/811272022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101029f0 0x7fdc10102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/811272022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10101c20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 -- 172.21.15.96:0/811272022 >> 172.21.15.96:0/811272022 conn(0x7fdc100fcd90 msgr2=0x7fdc100ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 -- 172.21.15.96:0/811272022 shutdown_connections 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.440+0000 7fdc17ed1640 1 -- 172.21.15.96:0/811272022 wait complete. 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 Processor -- start 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 -- start start 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101029f0 0x7fdc10198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101033b0 0x7fdc1019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d480 con 0x7fdc10101800 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d300 con 0x7fdc101033b0 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc17ed1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1010d600 con 0x7fdc101029f0 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43404/0 (socket says 172.21.15.96:43404) 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 -- 172.21.15.96:0/1774789830 learned_addr learned my addr 172.21.15.96:0/1774789830 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc1646e640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101033b0 0x7fdc1019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc1546c640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101029f0 0x7fdc10198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 -- 172.21.15.96:0/1774789830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101029f0 msgr2=0x7fdc10198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101029f0 0x7fdc10198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 -- 172.21.15.96:0/1774789830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101033b0 msgr2=0x7fdc1019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101033b0 0x7fdc1019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.441+0000 7fdc15c6d640 1 -- 172.21.15.96:0/1774789830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1019f710 con 0x7fdc10101800 2024-03-18T15:44:59.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc15c6d640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c000c00 tx=0x7fdc0c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c041070 con 0x7fdc10101800 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc0c004450 con 0x7fdc10101800 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1019f9a0 con 0x7fdc10101800 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c034440 con 0x7fdc10101800 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.442+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1019fe90 con 0x7fdc10101800 2024-03-18T15:44:59.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.443+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc10108b20 con 0x7fdc10101800 2024-03-18T15:44:59.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.444+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fdc0c044070 con 0x7fdc10101800 2024-03-18T15:44:59.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.444+0000 7fdc06ffd640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 0x7fdbfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.444+0000 7fdc1546c640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 0x7fdbfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.444+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fdc0c0f0550 con 0x7fdc10101800 2024-03-18T15:44:59.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.445+0000 7fdc1546c640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 0x7fdbfc0a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc000095f0 tx=0x7fdc00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.449+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fdc0c0bd000 con 0x7fdc10101800 2024-03-18T15:44:59.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.666+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc1002d3f0 con 0x7fdc10101800 2024-03-18T15:44:59.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.667+0000 7fdc06ffd640 1 -- 172.21.15.96:0/1774789830 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11942 (secure 0 0 0) 0x7fdc0c0c10a0 con 0x7fdc10101800 2024-03-18T15:44:59.667 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:44:59.668 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:44:18.699310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:44:59.668 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 33 2024-03-18T15:44:59.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 msgr2=0x7fdbfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 0x7fdbfc0a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc000095f0 tx=0x7fdc00005f90 comp rx=0 tx=0).stop 2024-03-18T15:44:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 msgr2=0x7fdc10197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c000c00 tx=0x7fdc0c02c640 comp rx=0 tx=0).stop 2024-03-18T15:44:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 shutdown_connections 2024-03-18T15:44:59.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fdbfc0a2800 0x7fdbfc0a4cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc101033b0 0x7fdc1019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc101029f0 0x7fdc10198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 --2- 172.21.15.96:0/1774789830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc10101800 0x7fdc10197af0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 >> 172.21.15.96:0/1774789830 conn(0x7fdc100fcd90 msgr2=0x7fdc10105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 shutdown_connections 2024-03-18T15:44:59.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.669+0000 7fdc17ed1640 1 -- 172.21.15.96:0/1774789830 wait complete. 2024-03-18T15:44:59.681 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:44:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 -- 172.21.15.96:0/3968623887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce94101810 msgr2=0x7fce94101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 --2- 172.21.15.96:0/3968623887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce94101810 0x7fce94101c30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fce84009900 tx=0x7fce8402bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 -- 172.21.15.96:0/3968623887 shutdown_connections 2024-03-18T15:44:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 --2- 172.21.15.96:0/3968623887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce941033c0 0x7fce94109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 --2- 172.21.15.96:0/3968623887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94102a00 0x7fce94102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 --2- 172.21.15.96:0/3968623887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce94101810 0x7fce94101c30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 -- 172.21.15.96:0/3968623887 >> 172.21.15.96:0/3968623887 conn(0x7fce940fcd80 msgr2=0x7fce940ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 -- 172.21.15.96:0/3968623887 shutdown_connections 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.783+0000 7fce99230640 1 -- 172.21.15.96:0/3968623887 wait complete. 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 Processor -- start 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 -- start start 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94101810 0x7fce94197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce94102a00 0x7fce94198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410d480 con 0x7fce941033c0 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410d300 con 0x7fce94102a00 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce99230640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410d600 con 0x7fce94101810 2024-03-18T15:44:59.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43430/0 (socket says 172.21.15.96:43430) 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 -- 172.21.15.96:0/1113046979 learned_addr learned my addr 172.21.15.96:0/1113046979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce92d9d640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94101810 0x7fce94197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce8bfff640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce94102a00 0x7fce94198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 -- 172.21.15.96:0/1113046979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94101810 msgr2=0x7fce94197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94101810 0x7fce94197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 -- 172.21.15.96:0/1113046979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce94102a00 msgr2=0x7fce94198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce94102a00 0x7fce94198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.784+0000 7fce9359e640 1 -- 172.21.15.96:0/1113046979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9419f770 con 0x7fce941033c0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce9359e640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00eb10 tx=0x7fce8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce90d99640 1 -- 172.21.15.96:0/1113046979 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c004b20 con 0x7fce941033c0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9419fa60 con 0x7fce941033c0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce90d99640 1 -- 172.21.15.96:0/1113046979 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce8c004cc0 con 0x7fce941033c0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce90d99640 1 -- 172.21.15.96:0/1113046979 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c010760 con 0x7fce941033c0 2024-03-18T15:44:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.785+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9419ff60 con 0x7fce941033c0 2024-03-18T15:44:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce90d99640 1 -- 172.21.15.96:0/1113046979 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fce8c005020 con 0x7fce941033c0 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 Processor -- start 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 -- start start 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec7111840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ec706e490 con 0x562ec710f3b0 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ec7016c90 con 0x562ec7109850 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.787+0000 7fce9be16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ec706a530 con 0x562ec710a240 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:58352/0 (socket says 172.21.15.96:58352) 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 -- 172.21.15.96:0/897794117 learned_addr learned my addr 172.21.15.96:0/897794117 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce8bfff640 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec7111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce90d99640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 0x7fce800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 -- 172.21.15.96:0/897794117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 msgr2=0x562ec7109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 -- 172.21.15.96:0/897794117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 msgr2=0x562ec7111840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec7111840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce9359e640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 0x7fce800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 -- 172.21.15.96:0/897794117 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec70d1500 con 0x562ec710a240 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce90d99640 1 -- 172.21.15.96:0/1113046979 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fce8c0cf000 con 0x7fce941033c0 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce92d9d640 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce84009900 tx=0x7fce84009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80c92098a5d3c53a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce9359e640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 0x7fce800a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0049d0 tx=0x7fce8c06e6a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce897fa640 1 -- 172.21.15.96:0/897794117 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8402cb80 con 0x562ec710a240 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce897fa640 1 -- 172.21.15.96:0/897794117 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce84005040 con 0x562ec710a240 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.788+0000 7fce897fa640 1 -- 172.21.15.96:0/897794117 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84004920 con 0x562ec710a240 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 -- 172.21.15.96:0/897794117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 msgr2=0x562ec710ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fce84009900 tx=0x7fce84009560 comp rx=0 tx=0).stop 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 -- 172.21.15.96:0/897794117 shutdown_connections 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec7111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec710ed40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 --2- 172.21.15.96:0/897794117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 -- 172.21.15.96:0/897794117 >> 172.21.15.96:0/897794117 conn(0x562ec7104dc0 msgr2=0x562ec71051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 -- 172.21.15.96:0/897794117 shutdown_connections 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.789+0000 7fce9be16740 1 -- 172.21.15.96:0/897794117 wait complete. 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 -- start start 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 Processor -- start 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7113d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec71142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ec7112300 con 0x562ec710f3b0 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ec702a230 con 0x562ec7109850 2024-03-18T15:44:59.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ec7112480 con 0x562ec710a240 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce9be16740 10 client.0 _ll_register_callbacks cb 0x562ec7066010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec71142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43450/0 (socket says 172.21.15.96:43450) 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 -- 172.21.15.96:0/3188019238 learned_addr learned my addr 172.21.15.96:0/3188019238 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 -- 172.21.15.96:0/3188019238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 msgr2=0x562ec71142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec71142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 -- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 msgr2=0x562ec7113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7113d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 -- 172.21.15.96:0/3188019238 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec7039100 con 0x562ec710f3b0 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce92d9d640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec71142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:59.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce8bfff640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce7c011bf0 tx=0x7fce7c009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b698ae3ce94e5d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.790+0000 7fce7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00fd10 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fce7c004050 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce84005040 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fce7c028030 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 10 client.16722 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ec6d823f0 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce84005040 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fce7c0047a0 con 0x562ec710f3b0 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 10 client.16722 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 10 client.16722 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 20 client.16722 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec711fb70 0x562ec7121ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 4 client.16722 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:59.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ec707b9d0 con 0x562ec711fb70 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.791+0000 7fce92d9d640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec711fb70 0x562ec7121ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.792+0000 7fce7a7fc640 10 client.16722 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.792+0000 7fce92d9d640 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec711fb70 0x562ec7121ff0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=92c1368acac54742 server_cookie=8656d22942a61031 in_seq=0 out_seq=0 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.792+0000 7fce7a7fc640 1 -- 172.21.15.96:0/3188019238 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ec707b9d0 con 0x562ec711fb70 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.792+0000 7fce7a7fc640 10 client.16722 handle_command_reply: tid=0 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.792+0000 7fce9be16740 1 client.16722 shutdown 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 msgr2=0x562ec711b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce7c011bf0 tx=0x7fce7c009aa0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 shutdown_connections 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec711fb70 0x562ec7121ff0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710f3b0 0x562ec711b2a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce7a7fc640 0 client.16722 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec71142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 --2- 172.21.15.96:0/3188019238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec7109850 0x562ec7113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 >> 172.21.15.96:0/3188019238 conn(0x562ec7104dc0 msgr2=0x562ec7103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 shutdown_connections 2024-03-18T15:44:59.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 1 -- 172.21.15.96:0/3188019238 wait complete. 2024-03-18T15:44:59.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.793+0000 7fce9be16740 20 client.16722 trim_cache size 0 max 16384 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 Processor -- start 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 -- start start 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec7103c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce941a8d40 con 0x562ec7109850 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce941aa940 con 0x562ec710f3b0 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce9be16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce941a9410 con 0x562ec710a240 2024-03-18T15:44:59.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43456/0 (socket says 172.21.15.96:43456) 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 -- 172.21.15.96:0/2488664004 learned_addr learned my addr 172.21.15.96:0/2488664004 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce92d9d640 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec7103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 -- 172.21.15.96:0/2488664004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 msgr2=0x562ec7103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec7103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 -- 172.21.15.96:0/2488664004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 msgr2=0x562ec71041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.806+0000 7fce8bfff640 1 -- 172.21.15.96:0/2488664004 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec6e4c7d0 con 0x562ec7109850 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce8bfff640 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fce7c011fd0 tx=0x7fce7c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852caa23d9fbace8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce88ff9640 1 -- 172.21.15.96:0/2488664004 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c000e60 con 0x562ec7109850 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce88ff9640 1 -- 172.21.15.96:0/2488664004 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c011660 con 0x562ec7109850 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce88ff9640 1 -- 172.21.15.96:0/2488664004 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c004040 con 0x562ec7109850 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 -- 172.21.15.96:0/2488664004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 msgr2=0x562ec7103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fce7c011fd0 tx=0x7fce7c005f90 comp rx=0 tx=0).stop 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 -- 172.21.15.96:0/2488664004 shutdown_connections 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec710a240 0x562ec7103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 --2- 172.21.15.96:0/2488664004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec7109850 0x562ec7103750 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 -- 172.21.15.96:0/2488664004 >> 172.21.15.96:0/2488664004 conn(0x562ec7104dc0 msgr2=0x562ec710b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.807+0000 7fce9be16740 1 -- 172.21.15.96:0/2488664004 shutdown_connections 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 -- 172.21.15.96:0/2488664004 wait complete. 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 -- start start 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 Processor -- start 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec7109850 0x562ec71813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71888c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce941aa270 con 0x562ec710a240 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce941a9ba0 con 0x562ec710f3b0 2024-03-18T15:44:59.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce941a6d20 con 0x562ec7109850 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce9be16740 10 client.0 _ll_register_callbacks cb 0x7fce7c011a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce8bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec7109850 0x562ec71813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43472/0 (socket says 172.21.15.96:43472) 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 -- 172.21.15.96:0/142573200 learned_addr learned my addr 172.21.15.96:0/142573200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce8bfff640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71888c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 -- 172.21.15.96:0/142573200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec7109850 msgr2=0x562ec71813b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec7109850 0x562ec71813b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 -- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 msgr2=0x562ec71888c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71888c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.808+0000 7fce92d9d640 1 -- 172.21.15.96:0/142573200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec7188eb0 con 0x562ec710a240 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce92d9d640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fce8402c170 tx=0x7fce8402cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23dfa3c9d1a900 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:59.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce84005d50 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fce84004020 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce7c011660 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fce84004240 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 10 client.16728 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ec718a1e0 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce7c011660 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fce8404e040 con 0x562ec710a240 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 10 client.16728 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 10 client.16728 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 20 client.16728 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec718a770 0x562ec718cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 4 client.16728 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.809+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ec7075d20 con 0x562ec718a770 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce8bfff640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec718a770 0x562ec718cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce7a7fc640 10 client.16728 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:44:59.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce8bfff640 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec718a770 0x562ec718cb90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0f8cab6d7a8ae8f server_cookie=c81f70dcefdb14a6 in_seq=0 out_seq=0 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce7a7fc640 1 -- 172.21.15.96:0/142573200 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ec7075d20 con 0x562ec718a770 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce7a7fc640 10 client.16728 handle_command_reply: tid=0 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 client.16728 shutdown 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 msgr2=0x562ec71818f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fce8402c170 tx=0x7fce8402cd80 comp rx=0 tx=0).stop 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 shutdown_connections 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x562ec718a770 0x562ec718cb90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ec710f3b0 0x562ec71888c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ec710a240 0x562ec71818f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce7a7fc640 0 client.16728 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 --2- 172.21.15.96:0/142573200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ec7109850 0x562ec71813b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 >> 172.21.15.96:0/142573200 conn(0x562ec7104dc0 msgr2=0x562ec7108020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 shutdown_connections 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 1 -- 172.21.15.96:0/142573200 wait complete. 2024-03-18T15:44:59.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.810+0000 7fce9be16740 20 client.16728 trim_cache size 0 max 16384 2024-03-18T15:44:59.818 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 msgr2=0x7fce800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 0x7fce800a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0049d0 tx=0x7fce8c06e6a0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 msgr2=0x7fce9419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00eb10 tx=0x7fce8c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 shutdown_connections 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fce800a2800 0x7fce800a4cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce941033c0 0x7fce9419f100 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce94102a00 0x7fce94198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 --2- 172.21.15.96:0/1113046979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce94101810 0x7fce94197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 >> 172.21.15.96:0/1113046979 conn(0x7fce940fcd80 msgr2=0x7fce94105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 shutdown_connections 2024-03-18T15:44:59.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.811+0000 7fce99230640 1 -- 172.21.15.96:0/1113046979 wait complete. 2024-03-18T15:44:59.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 -- 172.21.15.96:0/2928350374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 msgr2=0x7efe00101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 --2- 172.21.15.96:0/2928350374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe00101d60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009900 tx=0x7efdf002bc30 comp rx=0 tx=0).stop 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 -- 172.21.15.96:0/2928350374 shutdown_connections 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 --2- 172.21.15.96:0/2928350374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 0x7efe00108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 --2- 172.21.15.96:0/2928350374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe00101d60 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 --2- 172.21.15.96:0/2928350374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 0x7efe00100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 -- 172.21.15.96:0/2928350374 >> 172.21.15.96:0/2928350374 conn(0x7efe000fbc80 msgr2=0x7efe000fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 -- 172.21.15.96:0/2928350374 shutdown_connections 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.933+0000 7efe07405640 1 -- 172.21.15.96:0/2928350374 wait complete. 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 Processor -- start 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 -- start start 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 0x7efe00199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 0x7efe001a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010b270 con 0x7efe001018e0 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010b0f0 con 0x7efe001006f0 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe07405640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe0010b3f0 con 0x7efe001022a0 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43492/0 (socket says 172.21.15.96:43492) 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 -- 172.21.15.96:0/3343586511 learned_addr learned my addr 172.21.15.96:0/3343586511 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe059a2640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 0x7efe001a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe051a1640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 0x7efe00199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 -- 172.21.15.96:0/3343586511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 msgr2=0x7efe001a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 0x7efe001a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 -- 172.21.15.96:0/3343586511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 msgr2=0x7efe00199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 0x7efe00199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 -- 172.21.15.96:0/3343586511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe001a1910 con 0x7efe001018e0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.934+0000 7efe049a0640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009230 tx=0x7efdf0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.935+0000 7efdf67fc640 1 -- 172.21.15.96:0/3343586511 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0041070 con 0x7efe001018e0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.935+0000 7efdf67fc640 1 -- 172.21.15.96:0/3343586511 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf00044d0 con 0x7efe001018e0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.935+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe001a1ba0 con 0x7efe001018e0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.935+0000 7efdf67fc640 1 -- 172.21.15.96:0/3343586511 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0034440 con 0x7efe001018e0 2024-03-18T15:44:59.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.935+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe001a2090 con 0x7efe001018e0 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.937+0000 7efdf67fc640 1 -- 172.21.15.96:0/3343586511 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efdf0044070 con 0x7efe001018e0 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.937+0000 7efe09ff2740 1 Processor -- start 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.937+0000 7efe09ff2740 1 -- start start 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8829c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc8831840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560cc878e490 con 0x560cc8829850 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cc8736c90 con 0x560cc882a240 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe09ff2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cc878a530 con 0x560cc882f3b0 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe059a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8829c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe049a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc8831840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe059a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8829c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43500/0 (socket says 172.21.15.96:43500) 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38646/0 (socket says 172.21.15.96:38646) 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe059a2640 1 -- 172.21.15.96:0/2932322397 learned_addr learned my addr 172.21.15.96:0/2932322397 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efdf67fc640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 0x7efdec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 -- 172.21.15.96:0/2932322397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 msgr2=0x560cc8831840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc8831840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe059a2640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 0x7efdec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 -- 172.21.15.96:0/2932322397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 msgr2=0x560cc8829c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8829c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 -- 172.21.15.96:0/2932322397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc87f1500 con 0x560cc882a240 2024-03-18T15:44:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efdf67fc640 1 -- 172.21.15.96:0/3343586511 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efdf00f0700 con 0x7efe001018e0 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe051a1640 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe00101740 tx=0x7efdfc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102f63ea6dd4338a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efdeb7fe640 1 -- 172.21.15.96:0/2932322397 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc009d70 con 0x560cc882a240 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.938+0000 7efe059a2640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 0x7efdec0a4cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x560cc882aaa0 tx=0x7efdf8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efdeb7fe640 1 -- 172.21.15.96:0/2932322397 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc0035f0 con 0x560cc882a240 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efdeb7fe640 1 -- 172.21.15.96:0/2932322397 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc019650 con 0x560cc882a240 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 -- 172.21.15.96:0/2932322397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 msgr2=0x560cc882ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe00101740 tx=0x7efdfc011810 comp rx=0 tx=0).stop 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 -- 172.21.15.96:0/2932322397 shutdown_connections 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc8831840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc882ed40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2932322397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8829c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 -- 172.21.15.96:0/2932322397 >> 172.21.15.96:0/2932322397 conn(0x560cc8824dc0 msgr2=0x560cc88251c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 -- 172.21.15.96:0/2932322397 shutdown_connections 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.939+0000 7efe09ff2740 1 -- 172.21.15.96:0/2932322397 wait complete. 2024-03-18T15:44:59.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 -- start start 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 Processor -- start 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc88342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc883b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560cc8832360 con 0x560cc8829850 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cc874a230 con 0x560cc882a240 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cc88324e0 con 0x560cc882f3b0 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe09ff2740 10 client.0 _ll_register_callbacks cb 0x560cc8786010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43506/0 (socket says 172.21.15.96:43506) 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 -- 172.21.15.96:0/1698832482 learned_addr learned my addr 172.21.15.96:0/1698832482 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe049a0640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc88342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe059a2640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc883b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 -- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 msgr2=0x560cc883b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc883b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 -- 172.21.15.96:0/1698832482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 msgr2=0x560cc88342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc88342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.940+0000 7efe051a1640 1 -- 172.21.15.96:0/1698832482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc8759100 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe051a1640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efdfc01a000 tx=0x7efdfc004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf29694f92df8477 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc028020 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc0035f0 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efdfc016040 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdfc033050 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 10 client.16740 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560cc84a23f0 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc0035f0 con 0x560cc8829850 2024-03-18T15:44:59.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7efdfc0047e0 con 0x560cc8829850 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 10 client.16740 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 10 client.16740 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 20 client.16740 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc883f950 0x560cc8841dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 4 client.16740 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.941+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cc879b9d0 con 0x560cc883f950 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.942+0000 7efe049a0640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc883f950 0x560cc8841dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.942+0000 7efde8ff9640 10 client.16740 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.942+0000 7efe049a0640 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc883f950 0x560cc8841dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb62f5b32e596253 server_cookie=39c2d6946620c2f9 in_seq=0 out_seq=0 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efde8ff9640 1 -- 172.21.15.96:0/1698832482 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560cc879b9d0 con 0x560cc883f950 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efde8ff9640 10 client.16740 handle_command_reply: tid=0 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 client.16740 shutdown 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 msgr2=0x560cc8833d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efdfc01a000 tx=0x7efdfc004210 comp rx=0 tx=0).stop 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 shutdown_connections 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc883f950 0x560cc8841dc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882f3b0 0x560cc883b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efde8ff9640 0 client.16740 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882a240 0x560cc88342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 --2- 172.21.15.96:0/1698832482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8833d80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 >> 172.21.15.96:0/1698832482 conn(0x560cc8824dc0 msgr2=0x560cc8823fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 shutdown_connections 2024-03-18T15:44:59.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 1 -- 172.21.15.96:0/1698832482 wait complete. 2024-03-18T15:44:59.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.943+0000 7efe09ff2740 20 client.16740 trim_cache size 0 max 16384 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 Processor -- start 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 -- start start 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc883b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882f3b0 0x560cc88466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe001aafb0 con 0x560cc8829850 2024-03-18T15:44:59.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe001acbb0 con 0x560cc882f3b0 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe09ff2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe001ab680 con 0x560cc882a240 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe051a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe051a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43514/0 (socket says 172.21.15.96:43514) 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe051a1640 1 -- 172.21.15.96:0/2874457283 learned_addr learned my addr 172.21.15.96:0/2874457283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe049a0640 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc883b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.956+0000 7efe059a2640 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882f3b0 0x560cc88466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 -- 172.21.15.96:0/2874457283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 msgr2=0x560cc883b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc883b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 -- 172.21.15.96:0/2874457283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882f3b0 msgr2=0x560cc88466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882f3b0 0x560cc88466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 -- 172.21.15.96:0/2874457283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cc856c7d0 con 0x560cc8829850 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe051a1640 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0058c0 tx=0x7efdfc009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6366122ff8c3c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efdeaffd640 1 -- 172.21.15.96:0/2874457283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc03c070 con 0x560cc8829850 2024-03-18T15:44:59.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efdeaffd640 1 -- 172.21.15.96:0/2874457283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc016030 con 0x560cc8829850 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efdeaffd640 1 -- 172.21.15.96:0/2874457283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc028040 con 0x560cc8829850 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe09ff2740 1 -- 172.21.15.96:0/2874457283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 msgr2=0x560cc8823e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.957+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0058c0 tx=0x7efdfc009660 comp rx=0 tx=0).stop 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- 172.21.15.96:0/2874457283 shutdown_connections 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc882f3b0 0x560cc88466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc883b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- 172.21.15.96:0/2874457283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc8829850 0x560cc8823e60 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- 172.21.15.96:0/2874457283 >> 172.21.15.96:0/2874457283 conn(0x560cc8824dc0 msgr2=0x560cc8823750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- 172.21.15.96:0/2874457283 shutdown_connections 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- 172.21.15.96:0/2874457283 wait complete. 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- start start 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 Processor -- start 2024-03-18T15:44:59.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc8839390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe001ac4e0 con 0x560cc882f3b0 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe001abe10 con 0x560cc8829850 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe001a8f90 con 0x560cc882a240 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe09ff2740 10 client.0 _ll_register_callbacks cb 0x7efdf800bb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe059a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe051a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe051a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38658/0 (socket says 172.21.15.96:38658) 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe059a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43528/0 (socket says 172.21.15.96:43528) 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe051a1640 1 -- 172.21.15.96:0/4082412772 learned_addr learned my addr 172.21.15.96:0/4082412772 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.958+0000 7efe049a0640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc8839390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 -- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 msgr2=0x560cc8839390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc8839390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 -- 172.21.15.96:0/4082412772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 msgr2=0x560cc883a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 -- 172.21.15.96:0/4082412772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cc883a940 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe059a2640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe051a1640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efdfc004f70 tx=0x7efdfc003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f8331c65e9249c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efde8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc03c030 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efdfc023450 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc016030 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdfc028070 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe09ff2740 10 client.26552 fetch_fsmap learned FSMap version 33 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560cc88a6180 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.959+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdfc016030 con 0x560cc8829850 2024-03-18T15:44:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7efdfc04a070 con 0x560cc8829850 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 10 client.26552 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 10 client.26552 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 20 client.26552 populate_metadata read hostname 'smithi096' 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc88a69e0 0x560cc88a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 4 client.26552 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cc8795d20 con 0x560cc88a69e0 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe049a0640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc88a69e0 0x560cc88a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efde8ff9640 10 client.26552 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe049a0640 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc88a69e0 0x560cc88a8e00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df0a60a3c87750ae server_cookie=3ed7da369f0ffca9 in_seq=0 out_seq=0 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efde8ff9640 1 -- 172.21.15.96:0/4082412772 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cc8795d20 con 0x560cc88a69e0 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efde8ff9640 10 client.26552 handle_command_reply: tid=0 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 client.26552 shutdown 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 msgr2=0x560cc8838e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efdfc004f70 tx=0x7efdfc003b20 comp rx=0 tx=0).stop 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 shutdown_connections 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x560cc88a69e0 0x560cc88a8e00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efde8ff9640 0 client.26552 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.960+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560cc882f3b0 0x560cc883a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cc882a240 0x560cc8839390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 1 --2- 172.21.15.96:0/4082412772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cc8829850 0x560cc8838e50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 >> 172.21.15.96:0/4082412772 conn(0x560cc8824dc0 msgr2=0x560cc88288b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 shutdown_connections 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 1 -- 172.21.15.96:0/4082412772 wait complete. 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe09ff2740 20 client.26552 trim_cache size 0 max 16384 2024-03-18T15:44:59.969 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:44:59.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 msgr2=0x7efdec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 0x7efdec0a4cc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x560cc882aaa0 tx=0x7efdf8004ab0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 msgr2=0x7efe0019a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009230 tx=0x7efdf0005d50 comp rx=0 tx=0).stop 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 shutdown_connections 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efdec0a2800 0x7efdec0a4cc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe001022a0 0x7efe001a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe001018e0 0x7efe0019a240 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 --2- 172.21.15.96:0/3343586511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe001006f0 0x7efe00199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 >> 172.21.15.96:0/3343586511 conn(0x7efe000fbc80 msgr2=0x7efe00104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 shutdown_connections 2024-03-18T15:44:59.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:44:59.961+0000 7efe07405640 1 -- 172.21.15.96:0/3343586511 wait complete. 2024-03-18T15:44:59.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.079+0000 7ff8d9348640 1 -- 172.21.15.96:0/3758768805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41018e0 msgr2=0x7ff8d4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.079+0000 7ff8d9348640 1 --2- 172.21.15.96:0/3758768805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41018e0 0x7ff8d4101d60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009900 tx=0x7ff8bc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 -- 172.21.15.96:0/3758768805 shutdown_connections 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- 172.21.15.96:0/3758768805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41022a0 0x7ff8d4108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- 172.21.15.96:0/3758768805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41018e0 0x7ff8d4101d60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- 172.21.15.96:0/3758768805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41006f0 0x7ff8d4100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 -- 172.21.15.96:0/3758768805 >> 172.21.15.96:0/3758768805 conn(0x7ff8d40fbc80 msgr2=0x7ff8d40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 -- 172.21.15.96:0/3758768805 shutdown_connections 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 -- 172.21.15.96:0/3758768805 wait complete. 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 Processor -- start 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 -- start start 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41018e0 0x7ff8d4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.080+0000 7ff8d9348640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41022a0 0x7ff8d419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d9348640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410c370 con 0x7ff8d41006f0 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d9348640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410c1f0 con 0x7ff8d41018e0 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d9348640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410c4f0 con 0x7ff8d41022a0 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43542/0 (socket says 172.21.15.96:43542) 2024-03-18T15:45:00.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 -- 172.21.15.96:0/1974721847 learned_addr learned my addr 172.21.15.96:0/1974721847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41022a0 0x7ff8d419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41018e0 0x7ff8d4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 -- 172.21.15.96:0/1974721847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41022a0 msgr2=0x7ff8d419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41022a0 0x7ff8d419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 -- 172.21.15.96:0/1974721847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41018e0 msgr2=0x7ff8d4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41018e0 0x7ff8d4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 -- 172.21.15.96:0/1974721847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d419f730 con 0x7ff8d41006f0 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e9c0 tx=0x7ff8c400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d0ff9640 1 -- 172.21.15.96:0/1974721847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400cd80 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d0ff9640 1 -- 172.21.15.96:0/1974721847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8c40035f0 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d419fa20 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d0ff9640 1 -- 172.21.15.96:0/1974721847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4010690 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.081+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d419fef0 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8d0ff9640 1 -- 172.21.15.96:0/1974721847 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff8c40037a0 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8d0ff9640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 0x7ff8c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8d0ff9640 1 -- 172.21.15.96:0/1974721847 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff8c40ceea0 con 0x7ff8d41006f0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 0x7ff8c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8dc00c740 1 Processor -- start 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.084+0000 7ff8dc00c740 1 -- start start 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb917c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 0x5572cb91f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572cb87c490 con 0x5572cb918240 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572cb824c90 con 0x5572cb917850 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572cb878530 con 0x5572cb91d3b0 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43548/0 (socket says 172.21.15.96:43548) 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/3057169909 learned_addr learned my addr 172.21.15.96:0/3057169909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb917c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 0x7ff8c00a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 0x5572cb91f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/3057169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 msgr2=0x5572cb91f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 0x5572cb91f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/3057169909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 msgr2=0x5572cb917c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb917c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/3057169909 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572cb8df500 con 0x5572cb918240 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 0x5572cb91f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb917c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.085+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc036040 tx=0x7ff8bc03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6bf85ae14e13ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8c97fa640 1 -- 172.21.15.96:0/3057169909 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc005040 con 0x5572cb918240 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8c97fa640 1 -- 172.21.15.96:0/3057169909 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8bc04f070 con 0x5572cb918240 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8c97fa640 1 -- 172.21.15.96:0/3057169909 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc003850 con 0x5572cb918240 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3057169909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 msgr2=0x5572cb91cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc036040 tx=0x7ff8bc03d000 comp rx=0 tx=0).stop 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3057169909 shutdown_connections 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb91d3b0 0x5572cb91f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb91cd40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3057169909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb917c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.086+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3057169909 >> 172.21.15.96:0/3057169909 conn(0x5572cb912dc0 msgr2=0x5572cb9131c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3057169909 shutdown_connections 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3057169909 wait complete. 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- start start 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 Processor -- start 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb921cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb91d3b0 0x5572cb929220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572cb920300 con 0x5572cb918240 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5572cb838230 con 0x5572cb91d3b0 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572cb920480 con 0x5572cb917850 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8dc00c740 10 client.0 _ll_register_callbacks cb 0x5572cb874010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43562/0 (socket says 172.21.15.96:43562) 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/155658470 learned_addr learned my addr 172.21.15.96:0/155658470 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb921cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb91d3b0 0x5572cb929220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 msgr2=0x5572cb921cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb921cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/155658470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb91d3b0 msgr2=0x5572cb929220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb91d3b0 0x5572cb929220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.087+0000 7ff8ca7fc640 1 -- 172.21.15.96:0/155658470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572cb847100 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc02c650 tx=0x7ff8bc004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d207c1042e063f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc005450 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff8bc0055f0 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8bc04f070 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc061030 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 10 client.16758 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572cb5903f0 con 0x5572cb918240 2024-03-18T15:45:00.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8bc04f070 con 0x5572cb918240 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff8bc034730 con 0x5572cb918240 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 10 client.16758 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 10 client.16758 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 20 client.16758 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb92d680 0x5572cb92fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 4 client.16758 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.088+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572cb8899d0 con 0x5572cb92d680 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.089+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb92d680 0x5572cb92fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.089+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb92d680 0x5572cb92fa70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cb048e480436539 server_cookie=4fa5d1d760514717 in_seq=0 out_seq=0 2024-03-18T15:45:00.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.089+0000 7ff8ba7fc640 10 client.16758 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:45:00.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/155658470 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5572cb8899d0 con 0x5572cb92d680 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8ba7fc640 10 client.16758 handle_command_reply: tid=0 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 client.16758 shutdown 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 msgr2=0x5572cb9221f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc02c650 tx=0x7ff8bc004e80 comp rx=0 tx=0).stop 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 shutdown_connections 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb92d680 0x5572cb92fa70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb91d3b0 0x5572cb929220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb918240 0x5572cb9221f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8ba7fc640 0 client.16758 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/155658470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb921cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 >> 172.21.15.96:0/155658470 conn(0x5572cb912dc0 msgr2=0x5572cb911fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 shutdown_connections 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 1 -- 172.21.15.96:0/155658470 wait complete. 2024-03-18T15:45:00.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.091+0000 7ff8dc00c740 20 client.16758 trim_cache size 0 max 16384 2024-03-18T15:45:00.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.104+0000 7ff8dc00c740 1 Processor -- start 2024-03-18T15:45:00.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.104+0000 7ff8dc00c740 1 -- start start 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb9212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb918240 0x5572cb9291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41a8cc0 con 0x5572cb91d3b0 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41aa8c0 con 0x5572cb918240 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41a9390 con 0x5572cb917850 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43578/0 (socket says 172.21.15.96:43578) 2024-03-18T15:45:00.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 -- 172.21.15.96:0/4006185013 learned_addr learned my addr 172.21.15.96:0/4006185013 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb918240 0x5572cb9291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb9212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 -- 172.21.15.96:0/4006185013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 msgr2=0x5572cb9212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb9212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 -- 172.21.15.96:0/4006185013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb918240 msgr2=0x5572cb9291e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb918240 0x5572cb9291e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 -- 172.21.15.96:0/4006185013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572cb65a7d0 con 0x5572cb91d3b0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc012590 tx=0x7ff8cc012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec97ab6c00ae832 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8c8ff9640 1 -- 172.21.15.96:0/4006185013 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc010550 con 0x5572cb91d3b0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8c8ff9640 1 -- 172.21.15.96:0/4006185013 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8cc010bd0 con 0x5572cb91d3b0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.105+0000 7ff8c8ff9640 1 -- 172.21.15.96:0/4006185013 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc01bdf0 con 0x5572cb91d3b0 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- 172.21.15.96:0/4006185013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 msgr2=0x5572cb911750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc012590 tx=0x7ff8cc012a50 comp rx=0 tx=0).stop 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- 172.21.15.96:0/4006185013 shutdown_connections 2024-03-18T15:45:00.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb911750 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb918240 0x5572cb9291e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/4006185013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb917850 0x5572cb9212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- 172.21.15.96:0/4006185013 >> 172.21.15.96:0/4006185013 conn(0x5572cb912dc0 msgr2=0x5572cb921cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- 172.21.15.96:0/4006185013 shutdown_connections 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- 172.21.15.96:0/4006185013 wait complete. 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 -- start start 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.106+0000 7ff8dc00c740 1 Processor -- start 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb926b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb918240 0x5572cb927040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41aa1f0 con 0x5572cb91d3b0 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41a9b20 con 0x5572cb917850 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d41a6ca0 con 0x5572cb918240 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 10 client.0 _ll_register_callbacks cb 0x7ff8c4009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43592/0 (socket says 172.21.15.96:43592) 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb926b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 -- 172.21.15.96:0/3217697348 learned_addr learned my addr 172.21.15.96:0/3217697348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8ca7fc640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb918240 0x5572cb927040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 -- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb918240 msgr2=0x5572cb927040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb918240 0x5572cb927040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 -- 172.21.15.96:0/3217697348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 msgr2=0x5572cb926b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb926b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 -- 172.21.15.96:0/3217697348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572cb9285a0 con 0x5572cb91d3b0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb926b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8d37fe640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000f80 tx=0x7ff8cc005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e605b4e4693af5a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc02a020 con 0x5572cb91d3b0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff8cc010750 con 0x5572cb91d3b0 2024-03-18T15:45:00.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.107+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8cc010bd0 con 0x5572cb91d3b0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff8cc01b690 con 0x5572cb91d3b0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 10 client.16764 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5572cb993ac0 con 0x5572cb91d3b0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8cc010bd0 con 0x5572cb91d3b0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff8cc01a070 con 0x5572cb91d3b0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 10 client.16764 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 10 client.16764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 20 client.16764 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb9945a0 0x5572cb9969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 4 client.16764 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572cb921aa0 con 0x5572cb9945a0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb9945a0 0x5572cb9969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8d2ffd640 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb9945a0 0x5572cb9969c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79cefd8ca19636f3 server_cookie=bb362b5b55db282c in_seq=0 out_seq=0 2024-03-18T15:45:00.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.108+0000 7ff8ba7fc640 10 client.16764 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8ba7fc640 1 -- 172.21.15.96:0/3217697348 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572cb921aa0 con 0x5572cb9945a0 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8ba7fc640 10 client.16764 handle_command_reply: tid=0 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 client.16764 shutdown 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 msgr2=0x5572cb927fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000f80 tx=0x7ff8cc005a50 comp rx=0 tx=0).stop 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 shutdown_connections 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5572cb9945a0 0x5572cb9969c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572cb91d3b0 0x5572cb927fb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572cb918240 0x5572cb927040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 --2- 172.21.15.96:0/3217697348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5572cb917850 0x5572cb926b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8ba7fc640 0 client.16764 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 >> 172.21.15.96:0/3217697348 conn(0x5572cb912dc0 msgr2=0x5572cb916020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 shutdown_connections 2024-03-18T15:45:00.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 1 -- 172.21.15.96:0/3217697348 wait complete. 2024-03-18T15:45:00.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.111+0000 7ff8dc00c740 20 client.16764 trim_cache size 0 max 16384 2024-03-18T15:45:00.117 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 msgr2=0x7ff8c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 0x7ff8c00a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc006010 tx=0x7ff8bc000bc0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 msgr2=0x7ff8d4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400e9c0 tx=0x7ff8c400ee80 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 shutdown_connections 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff8c00a2800 0x7ff8c00a4cc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8d41022a0 0x7ff8d419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8d41018e0 0x7ff8d4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 --2- 172.21.15.96:0/1974721847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8d41006f0 0x7ff8d4197b10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 >> 172.21.15.96:0/1974721847 conn(0x7ff8d40fbc80 msgr2=0x7ff8d40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 shutdown_connections 2024-03-18T15:45:00.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.112+0000 7ff8d9348640 1 -- 172.21.15.96:0/1974721847 wait complete. 2024-03-18T15:45:00.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- 172.21.15.96:0/1118504752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be01022b0 msgr2=0x7f2be0108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 --2- 172.21.15.96:0/1118504752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be01022b0 0x7f2be0108b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009ca0 tx=0x7f2bd002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- 172.21.15.96:0/1118504752 shutdown_connections 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 --2- 172.21.15.96:0/1118504752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be01022b0 0x7f2be0108b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 --2- 172.21.15.96:0/1118504752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01018f0 0x7f2be0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 --2- 172.21.15.96:0/1118504752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be0100700 0x7f2be0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- 172.21.15.96:0/1118504752 >> 172.21.15.96:0/1118504752 conn(0x7f2be00fbc90 msgr2=0x7f2be00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- 172.21.15.96:0/1118504752 shutdown_connections 2024-03-18T15:45:00.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- 172.21.15.96:0/1118504752 wait complete. 2024-03-18T15:45:00.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 Processor -- start 2024-03-18T15:45:00.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.238+0000 7f2be4836640 1 -- start start 2024-03-18T15:45:00.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01018f0 0x7f2be0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2be010c380 con 0x7f2be01022b0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be010c200 con 0x7f2be0100700 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2be4836640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2be010c500 con 0x7f2be01018f0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43610/0 (socket says 172.21.15.96:43610) 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38744/0 (socket says 172.21.15.96:38744) 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 -- 172.21.15.96:0/2784603680 learned_addr learned my addr 172.21.15.96:0/2784603680 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01018f0 0x7f2be0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 -- 172.21.15.96:0/2784603680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01018f0 msgr2=0x7f2be0196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01018f0 0x7f2be0196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 -- 172.21.15.96:0/2784603680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 msgr2=0x7f2be01969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 -- 172.21.15.96:0/2784603680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be019e610 con 0x7f2be01022b0 2024-03-18T15:45:00.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bde59c640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bded9d640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0037040 tx=0x7f2bd0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.239+0000 7f2bcf7fe640 1 -- 172.21.15.96:0/2784603680 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0041020 con 0x7f2be01022b0 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.240+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be019e8a0 con 0x7f2be01022b0 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.240+0000 7f2bcf7fe640 1 -- 172.21.15.96:0/2784603680 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0034cd0 con 0x7f2be01022b0 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.240+0000 7f2bcf7fe640 1 -- 172.21.15.96:0/2784603680 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0039040 con 0x7f2be01022b0 2024-03-18T15:45:00.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.240+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be0069b40 con 0x7f2be01022b0 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.242+0000 7f2be7427740 1 Processor -- start 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bcf7fe640 1 -- 172.21.15.96:0/2784603680 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2bd0044020 con 0x7f2be01022b0 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 -- start start 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x56341601ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416026840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563415f83490 con 0x56341601f240 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563415f2bc90 con 0x5634160243b0 2024-03-18T15:45:00.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2be7427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563415f7f530 con 0x56341601e850 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43612/0 (socket says 172.21.15.96:43612) 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bde59c640 1 -- 172.21.15.96:0/1529505368 learned_addr learned my addr 172.21.15.96:0/1529505368 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x56341601ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bcf7fe640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 0x7f2bc80a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416026840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.243+0000 7f2bded9d640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 0x7f2bc80a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 -- 172.21.15.96:0/1529505368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 msgr2=0x56341601ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x56341601ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 -- 172.21.15.96:0/1529505368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 msgr2=0x563416026840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416026840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 -- 172.21.15.96:0/1529505368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563415fe6500 con 0x56341601f240 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bcf7fe640 1 -- 172.21.15.96:0/2784603680 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2bd00f0920 con 0x7f2be01022b0 2024-03-18T15:45:00.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bde59c640 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80114f0 tx=0x7f2bd80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e51375af2e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bded9d640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 0x7f2bc80a6ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b9cf0 tx=0x7f2bd00b2800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bccff9640 1 -- 172.21.15.96:0/1529505368 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801e070 con 0x56341601f240 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bccff9640 1 -- 172.21.15.96:0/1529505368 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd80035f0 con 0x56341601f240 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.244+0000 7f2bccff9640 1 -- 172.21.15.96:0/1529505368 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd800f7a0 con 0x56341601f240 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- 172.21.15.96:0/1529505368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 msgr2=0x563416023d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80114f0 tx=0x7f2bd80119b0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- 172.21.15.96:0/1529505368 shutdown_connections 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416026840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416023d40 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- 172.21.15.96:0/1529505368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x56341601ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- 172.21.15.96:0/1529505368 >> 172.21.15.96:0/1529505368 conn(0x563416019dc0 msgr2=0x56341601a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- 172.21.15.96:0/1529505368 shutdown_connections 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- 172.21.15.96:0/1529505368 wait complete. 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- start start 2024-03-18T15:45:00.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 Processor -- start 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601f240 0x5634160292e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416030320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563416027360 con 0x56341601e850 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563415f3f230 con 0x5634160243b0 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634160274e0 con 0x56341601f240 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.245+0000 7f2be7427740 10 client.0 _ll_register_callbacks cb 0x563415f7b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43628/0 (socket says 172.21.15.96:43628) 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 -- 172.21.15.96:0/4027524450 learned_addr learned my addr 172.21.15.96:0/4027524450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bde59c640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601f240 0x5634160292e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416030320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 -- 172.21.15.96:0/4027524450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601f240 msgr2=0x5634160292e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601f240 0x5634160292e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 -- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 msgr2=0x563416030320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416030320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 -- 172.21.15.96:0/4027524450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563415f4e100 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40045c0 tx=0x7f2bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54539e732a420356 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4025060 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2bd4011880 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd80035f0 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2bd4011a80 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2be7427740 10 client.16776 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563415c973f0 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.246+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd80035f0 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2bd402bca0 con 0x56341601e850 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 10 client.16776 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 10 client.16776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 20 client.16776 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563416034be0 0x563416037080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 4 client.16776 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563415f909d0 con 0x563416034be0 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2bde59c640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563416034be0 0x563416037080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2bc5ffb640 10 client.16776 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.247+0000 7f2bde59c640 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563416034be0 0x563416037080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58a855fa44a96b73 server_cookie=df579c14e43552a7 in_seq=0 out_seq=0 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/4027524450 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563415f909d0 con 0x563416034be0 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2bc5ffb640 10 client.16776 handle_command_reply: tid=0 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 client.16776 shutdown 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 msgr2=0x563416028da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40045c0 tx=0x7f2bd40046c0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 shutdown_connections 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x563416034be0 0x563416037080 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416030320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2bc5ffb640 0 client.16776 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601f240 0x5634160292e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 --2- 172.21.15.96:0/4027524450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601e850 0x563416028da0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 >> 172.21.15.96:0/4027524450 conn(0x563416019dc0 msgr2=0x563416018fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 shutdown_connections 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 1 -- 172.21.15.96:0/4027524450 wait complete. 2024-03-18T15:45:00.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.248+0000 7f2be7427740 20 client.16776 trim_cache size 0 max 16384 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 Processor -- start 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 -- start start 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416029240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416018750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a6bc0 con 0x56341601f240 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a87c0 con 0x5634160243b0 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2be7427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a7290 con 0x56341601e850 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.261+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43638/0 (socket says 172.21.15.96:43638) 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 -- 172.21.15.96:0/910120804 learned_addr learned my addr 172.21.15.96:0/910120804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416029240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416018750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 -- 172.21.15.96:0/910120804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 msgr2=0x563416029240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416029240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 -- 172.21.15.96:0/910120804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 msgr2=0x563416018750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416018750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 -- 172.21.15.96:0/910120804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563415d617d0 con 0x56341601f240 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416029240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bde59c640 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2bd801c040 tx=0x7f2bd8003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5210c47dcd66a133 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bc7fff640 1 -- 172.21.15.96:0/910120804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd801e040 con 0x56341601f240 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bc7fff640 1 -- 172.21.15.96:0/910120804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8009bf0 con 0x56341601f240 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.262+0000 7f2bc7fff640 1 -- 172.21.15.96:0/910120804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd802b440 con 0x56341601f240 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- 172.21.15.96:0/910120804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 msgr2=0x56341601d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2bd801c040 tx=0x7f2bd8003fb0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- 172.21.15.96:0/910120804 shutdown_connections 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x563416018750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x56341601d730 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- 172.21.15.96:0/910120804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416029240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- 172.21.15.96:0/910120804 >> 172.21.15.96:0/910120804 conn(0x563416019dc0 msgr2=0x56341601d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- 172.21.15.96:0/910120804 shutdown_connections 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- 172.21.15.96:0/910120804 wait complete. 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- start start 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 Processor -- start 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x56341609d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a80f0 con 0x56341601f240 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a7a20 con 0x5634160243b0 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01a4ba0 con 0x56341601e850 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2be7427740 10 client.0 _ll_register_callbacks cb 0x7f2bd40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43646/0 (socket says 172.21.15.96:43646) 2024-03-18T15:45:00.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 -- 172.21.15.96:0/3933356922 learned_addr learned my addr 172.21.15.96:0/3933356922 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x56341609d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 -- 172.21.15.96:0/3933356922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 msgr2=0x563416095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416095df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 -- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 msgr2=0x56341609d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x56341609d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bde59c640 1 -- 172.21.15.96:0/3933356922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56341609d960 con 0x56341601f240 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.263+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x56341609d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bde59c640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8012ec0 tx=0x7f2bd8003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16694038567facb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8038020 con 0x56341601f240 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2bd80037b0 con 0x56341601f240 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd8009bf0 con 0x56341601f240 2024-03-18T15:45:00.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8043050 con 0x56341601f240 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 10 client.16782 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56341609ec60 con 0x56341601f240 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8009bf0 con 0x56341601f240 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2bd802b300 con 0x56341601f240 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 10 client.16782 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 10 client.16782 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 20 client.16782 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56341609f4b0 0x5634160a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 4 client.16782 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563415f8ad20 con 0x56341609f4b0 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.264+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56341609f4b0 0x5634160a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2bddd9b640 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56341609f4b0 0x5634160a18d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb783c1817856a9b server_cookie=6d7cfbf51b10ec5f in_seq=0 out_seq=0 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2bc5ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2bc5ffb640 1 -- 172.21.15.96:0/3933356922 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563415f8ad20 con 0x56341609f4b0 2024-03-18T15:45:00.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2bc5ffb640 10 client.16782 handle_command_reply: tid=0 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 client.16782 shutdown 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 msgr2=0x563416096330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8012ec0 tx=0x7f2bd8003c50 comp rx=0 tx=0).stop 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 shutdown_connections 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x56341609f4b0 0x5634160a18d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5634160243b0 0x56341609d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56341601f240 0x563416096330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2bc5ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 --2- 172.21.15.96:0/3933356922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56341601e850 0x563416095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 >> 172.21.15.96:0/3933356922 conn(0x563416019dc0 msgr2=0x5634160290c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 shutdown_connections 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 1 -- 172.21.15.96:0/3933356922 wait complete. 2024-03-18T15:45:00.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.265+0000 7f2be7427740 20 client.16782 trim_cache size 0 max 16384 2024-03-18T15:45:00.276 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 msgr2=0x7f2bc80a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 0x7f2bc80a6ed0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00b9cf0 tx=0x7f2bd00b2800 comp rx=0 tx=0).stop 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 msgr2=0x7f2be019dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0037040 tx=0x7f2bd0032f40 comp rx=0 tx=0).stop 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 shutdown_connections 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2bc80a4a10 0x7f2bc80a6ed0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be01022b0 0x7f2be019dfa0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be01018f0 0x7f2be0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 --2- 172.21.15.96:0/2784603680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2be0100700 0x7f2be01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 >> 172.21.15.96:0/2784603680 conn(0x7f2be00fbc90 msgr2=0x7f2be0104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 shutdown_connections 2024-03-18T15:45:00.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.266+0000 7f2be4836640 1 -- 172.21.15.96:0/2784603680 wait complete. 2024-03-18T15:45:00.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 -- 172.21.15.96:0/3083113892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c70100a10 msgr2=0x7f7c70100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3083113892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c70100a10 0x7f7c70100e70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c6402bc30 comp rx=0 tx=0).stop 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 -- 172.21.15.96:0/3083113892 shutdown_connections 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3083113892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 0x7f7c70101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3083113892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c70100a10 0x7f7c70100e70 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3083113892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c701000b0 0x7f7c701004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.382+0000 7f7c757b9640 1 -- 172.21.15.96:0/3083113892 >> 172.21.15.96:0/3083113892 conn(0x7f7c700fbc70 msgr2=0x7f7c700fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- 172.21.15.96:0/3083113892 shutdown_connections 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- 172.21.15.96:0/3083113892 wait complete. 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 Processor -- start 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- start start 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 0x7f7c7019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 0x7f7c701a1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010f620 con 0x7f7c701000b0 2024-03-18T15:45:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010f4a0 con 0x7f7c70100a10 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c757b9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7010f7a0 con 0x7f7c701013b0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.383+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43658/0 (socket says 172.21.15.96:43658) 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 -- 172.21.15.96:0/3042950687 learned_addr learned my addr 172.21.15.96:0/3042950687 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 0x7f7c7019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 0x7f7c701a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 -- 172.21.15.96:0/3042950687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 msgr2=0x7f7c701a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 0x7f7c701a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 -- 172.21.15.96:0/3042950687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 msgr2=0x7f7c7019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 0x7f7c7019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 -- 172.21.15.96:0/3042950687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c701a1930 con 0x7f7c701000b0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 0x7f7c7019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c6effd640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c57fff640 1 -- 172.21.15.96:0/3042950687 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800cd80 con 0x7f7c701000b0 2024-03-18T15:45:00.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c701a1c20 con 0x7f7c701000b0 2024-03-18T15:45:00.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c701a21c0 con 0x7f7c701000b0 2024-03-18T15:45:00.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c57fff640 1 -- 172.21.15.96:0/3042950687 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c68010910 con 0x7f7c701000b0 2024-03-18T15:45:00.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.384+0000 7f7c57fff640 1 -- 172.21.15.96:0/3042950687 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68026060 con 0x7f7c701000b0 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.386+0000 7f7c57fff640 1 -- 172.21.15.96:0/3042950687 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7c68010ac0 con 0x7f7c701000b0 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 Processor -- start 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 -- start start 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c86fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c872840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d36c7cf490 con 0x55d36c86a850 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36c777c90 con 0x55d36c8703b0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c783a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d36c7cb530 con 0x55d36c86b240 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c57fff640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 0x7f7c580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c86fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43662/0 (socket says 172.21.15.96:43662) 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6f7fe640 1 -- 172.21.15.96:0/3964430192 learned_addr learned my addr 172.21.15.96:0/3964430192 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c872840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 0x7f7c580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/3964430192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 msgr2=0x55d36c86fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c86fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/3964430192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 msgr2=0x55d36c86ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c57fff640 1 -- 172.21.15.96:0/3042950687 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7c680cee90 con 0x7f7c701000b0 2024-03-18T15:45:00.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/3964430192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36c832500 con 0x55d36c8703b0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.387+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c872840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009230 tx=0x7f7c6402c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71911f5616fdcd98 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 0x7f7c580a4cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d36c86baa0 tx=0x7f7c60004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c557fa640 1 -- 172.21.15.96:0/3964430192 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64005d50 con 0x55d36c8703b0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c557fa640 1 -- 172.21.15.96:0/3964430192 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64005520 con 0x55d36c8703b0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c557fa640 1 -- 172.21.15.96:0/3964430192 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004c20 con 0x55d36c8703b0 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c783a7740 1 -- 172.21.15.96:0/3964430192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 msgr2=0x55d36c872840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.388+0000 7f7c783a7740 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c872840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009230 tx=0x7f7c6402c6d0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- 172.21.15.96:0/3964430192 shutdown_connections 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c872840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c86fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- 172.21.15.96:0/3964430192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c86ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- 172.21.15.96:0/3964430192 >> 172.21.15.96:0/3964430192 conn(0x55d36c865dc0 msgr2=0x55d36c8661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- 172.21.15.96:0/3964430192 shutdown_connections 2024-03-18T15:45:00.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- 172.21.15.96:0/3964430192 wait complete. 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- start start 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 Processor -- start 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c875210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d36c873300 con 0x55d36c86a850 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36c78b230 con 0x55d36c8703b0 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d36c873480 con 0x55d36c86b240 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c783a7740 10 client.0 _ll_register_callbacks cb 0x55d36c7c7010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c6effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43670/0 (socket says 172.21.15.96:43670) 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c6effd640 1 -- 172.21.15.96:0/1958566849 learned_addr learned my addr 172.21.15.96:0/1958566849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.389+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c875210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 -- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 msgr2=0x55d36c875210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c875210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 -- 172.21.15.96:0/1958566849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 msgr2=0x55d36c87c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 -- 172.21.15.96:0/1958566849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36c79a100 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c6effd640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c680cc900 tx=0x7f7c68097340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b67f9e180eefa39 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6809b390 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7c6809aca0 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c64005520 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c68099990 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 10 client.16794 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d36c4e33f0 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64005520 con 0x55d36c86a850 2024-03-18T15:45:00.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7c68097510 con 0x55d36c86a850 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 10 client.16794 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 20 client.16794 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c880bc0 0x55d36c883080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 4 client.16794 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.390+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d36c7dc9d0 con 0x55d36c880bc0 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.391+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c880bc0 0x55d36c883080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.391+0000 7f7c52ff5640 10 client.16794 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.391+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c880bc0 0x55d36c883080 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d102ade2c9c4d380 server_cookie=f31bce5680a3b402 in_seq=0 out_seq=0 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c52ff5640 1 -- 172.21.15.96:0/1958566849 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d36c7dc9d0 con 0x55d36c880bc0 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c52ff5640 10 client.16794 handle_command_reply: tid=0 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 client.16794 shutdown 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 msgr2=0x55d36c874cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7c680cc900 tx=0x7f7c68097340 comp rx=0 tx=0).stop 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 shutdown_connections 2024-03-18T15:45:00.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86b240 0x55d36c875210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c880bc0 0x55d36c883080 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1958566849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86a850 0x55d36c874cd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c52ff5640 0 client.16794 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 >> 172.21.15.96:0/1958566849 conn(0x55d36c865dc0 msgr2=0x55d36c864fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 shutdown_connections 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.392+0000 7f7c783a7740 1 -- 172.21.15.96:0/1958566849 wait complete. 2024-03-18T15:45:00.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.393+0000 7f7c783a7740 20 client.16794 trim_cache size 0 max 16384 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 Processor -- start 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 -- start start 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c874e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00e530 con 0x55d36c86b240 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c010130 con 0x55d36c8703b0 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c783a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00ec00 con 0x55d36c86a850 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43684/0 (socket says 172.21.15.96:43684) 2024-03-18T15:45:00.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/1481589218 learned_addr learned my addr 172.21.15.96:0/1481589218 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c874e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/1481589218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 msgr2=0x55d36c8748e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/1481589218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 msgr2=0x55d36c874e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c874e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.414+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/1481589218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36c5ad7d0 con 0x55d36c86b240 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.415+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7c64000c00 tx=0x7f7c64009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36334a31ea06be8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.415+0000 7f7c54ff9640 1 -- 172.21.15.96:0/1481589218 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64009520 con 0x55d36c86b240 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.415+0000 7f7c54ff9640 1 -- 172.21.15.96:0/1481589218 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64056020 con 0x55d36c86b240 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.415+0000 7f7c54ff9640 1 -- 172.21.15.96:0/1481589218 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c640035f0 con 0x55d36c86b240 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- 172.21.15.96:0/1481589218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 msgr2=0x55d36c87c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7c64000c00 tx=0x7f7c64009260 comp rx=0 tx=0).stop 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- 172.21.15.96:0/1481589218 shutdown_connections 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c874e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c87c200 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- 172.21.15.96:0/1481589218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8748e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- 172.21.15.96:0/1481589218 >> 172.21.15.96:0/1481589218 conn(0x55d36c865dc0 msgr2=0x55d36c86c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- 172.21.15.96:0/1481589218 shutdown_connections 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- 172.21.15.96:0/1481589218 wait complete. 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- start start 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 Processor -- start 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8793c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00fa60 con 0x55d36c86b240 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00f390 con 0x55d36c8703b0 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c00c510 con 0x55d36c86a850 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c783a7740 10 client.0 _ll_register_callbacks cb 0x7f7c6800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c6e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c6e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43690/0 (socket says 172.21.15.96:43690) 2024-03-18T15:45:00.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/200784340 learned_addr learned my addr 172.21.15.96:0/200784340 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.416+0000 7f7c6f7fe640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 msgr2=0x55d36c8793c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8793c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/200784340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 msgr2=0x55d36c87a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 -- 172.21.15.96:0/200784340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36c87af00 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c6e7fc640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c64005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf965ccb52f5c709 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c52ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64052170 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7c64004030 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c64056020 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7c64004250 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c783a7740 10 client.16800 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d36c87c200 con 0x55d36c86b240 2024-03-18T15:45:00.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.417+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64056020 con 0x55d36c86b240 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7c6404e040 con 0x55d36c86b240 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 10 client.16800 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 20 client.16800 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c8e7020 0x55d36c8e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 4 client.16800 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d36c7d6d20 con 0x55d36c8e7020 2024-03-18T15:45:00.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.418+0000 7f7c6effd640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c8e7020 0x55d36c8e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:00.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c52ff5640 10 client.16800 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:00.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c6effd640 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c8e7020 0x55d36c8e9410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7311b56d52f7386e server_cookie=3cad70cb4adb7edd in_seq=0 out_seq=0 2024-03-18T15:45:00.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c52ff5640 1 -- 172.21.15.96:0/200784340 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d36c7d6d20 con 0x55d36c8e7020 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c52ff5640 10 client.16800 handle_command_reply: tid=0 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 client.16800 shutdown 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 msgr2=0x55d36c879900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009900 tx=0x7f7c64005b50 comp rx=0 tx=0).stop 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 shutdown_connections 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36c8703b0 0x55d36c87a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d36c8e7020 0x55d36c8e9410 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d36c86b240 0x55d36c879900 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 --2- 172.21.15.96:0/200784340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d36c86a850 0x55d36c8793c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c52ff5640 0 client.16800 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 >> 172.21.15.96:0/200784340 conn(0x55d36c865dc0 msgr2=0x55d36c8748c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 shutdown_connections 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 1 -- 172.21.15.96:0/200784340 wait complete. 2024-03-18T15:45:00.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.419+0000 7f7c783a7740 20 client.16800 trim_cache size 0 max 16384 2024-03-18T15:45:00.426 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:45:00.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 msgr2=0x7f7c580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 0x7f7c580a4cc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x55d36c86baa0 tx=0x7f7c60004ab0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 msgr2=0x7f7c70199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800e9f0 tx=0x7f7c6800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 shutdown_connections 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7c580a2800 0x7f7c580a4cc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c701013b0 0x7f7c701a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7c70100a10 0x7f7c7019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 --2- 172.21.15.96:0/3042950687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c701000b0 0x7f7c70199d20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 >> 172.21.15.96:0/3042950687 conn(0x7f7c700fbc70 msgr2=0x7f7c700fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.420+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 shutdown_connections 2024-03-18T15:45:00.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:00.421+0000 7f7c757b9640 1 -- 172.21.15.96:0/3042950687 wait complete. 2024-03-18T15:45:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:00 smithi193 ceph-mon[36702]: pgmap v499: 97 pgs: 97 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 520 op/s 2024-03-18T15:45:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:00 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1774789830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:45:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:00 smithi096 ceph-mon[31930]: pgmap v499: 97 pgs: 97 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 520 op/s 2024-03-18T15:45:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:00 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1774789830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:45:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:00 smithi137 ceph-mon[35868]: pgmap v499: 97 pgs: 97 active+clean; 628 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 520 op/s 2024-03-18T15:45:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:00 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1774789830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:45:01.331 INFO:tasks.workunit.client.0.smithi096.stdout: 123 1161584 272 850457 133 662031 617 2024-03-18T15:45:01.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:45:01.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:45:01.437 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:45:01.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.557+0000 7f7106713640 1 -- 172.21.15.96:0/2211245942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001018d0 msgr2=0x7f7100101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.557+0000 7f7106713640 1 --2- 172.21.15.96:0/2211245942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001018d0 0x7f7100101d50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009900 tx=0x7f70f402bc30 comp rx=0 tx=0).stop 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- 172.21.15.96:0/2211245942 shutdown_connections 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- 172.21.15.96:0/2211245942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7100102290 0x7f7100108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- 172.21.15.96:0/2211245942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001018d0 0x7f7100101d50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- 172.21.15.96:0/2211245942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71001006e0 0x7f7100100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- 172.21.15.96:0/2211245942 >> 172.21.15.96:0/2211245942 conn(0x7f71000fbc70 msgr2=0x7f71000fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- 172.21.15.96:0/2211245942 shutdown_connections 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- 172.21.15.96:0/2211245942 wait complete. 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 Processor -- start 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- start start 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71001018d0 0x7f7100198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7100102290 0x7f710019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f710010c360 con 0x7f71001006e0 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f710010c1e0 con 0x7f71001018d0 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.558+0000 7f7106713640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f710010c4e0 con 0x7f7100102290 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43710/0 (socket says 172.21.15.96:43710) 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 -- 172.21.15.96:0/2460396098 learned_addr learned my addr 172.21.15.96:0/2460396098 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f7104cb0640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7100102290 0x7f710019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71001018d0 0x7f7100198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 -- 172.21.15.96:0/2460396098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7100102290 msgr2=0x7f710019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7100102290 0x7f710019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 -- 172.21.15.96:0/2460396098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71001018d0 msgr2=0x7f7100198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71001018d0 0x7f7100198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 -- 172.21.15.96:0/2460396098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710019f730 con 0x7f71001006e0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fffff640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9c0 tx=0x7f70f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fd7fa640 1 -- 172.21.15.96:0/2460396098 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000cd80 con 0x7f71001006e0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fd7fa640 1 -- 172.21.15.96:0/2460396098 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f00035f0 con 0x7f71001006e0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710019fa20 con 0x7f71001006e0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f70fd7fa640 1 -- 172.21.15.96:0/2460396098 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0010690 con 0x7f71001006e0 2024-03-18T15:45:01.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.559+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f710019fef0 con 0x7f71001006e0 2024-03-18T15:45:01.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.561+0000 7f70fd7fa640 1 -- 172.21.15.96:0/2460396098 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f70f00037a0 con 0x7f71001006e0 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.561+0000 7f71093d0740 1 Processor -- start 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.561+0000 7f71093d0740 1 -- start start 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f651380 0x55fd2f655e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f658980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f5b50d0 con 0x55fd2f650990 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f55dc70 con 0x55fd2f6564f0 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f71093d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f5b1170 con 0x55fd2f651380 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f70fd7fa640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 0x7f70e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43716/0 (socket says 172.21.15.96:43716) 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 -- 172.21.15.96:0/1093964735 learned_addr learned my addr 172.21.15.96:0/1093964735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f658980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f70fffff640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f651380 0x55fd2f655e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 0x7f70e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 -- 172.21.15.96:0/1093964735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f651380 msgr2=0x55fd2f655e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f651380 0x55fd2f655e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 -- 172.21.15.96:0/1093964735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 msgr2=0x55fd2f658980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f658980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f70fd7fa640 1 -- 172.21.15.96:0/2460396098 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f70f00cedf0 con 0x7f71001006e0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 -- 172.21.15.96:0/1093964735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2f3917e0 con 0x55fd2f650990 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f658980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.562+0000 7f7104cb0640 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005c40 tx=0x7f70f800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7780c1e170ea1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f70e67fc640 1 -- 172.21.15.96:0/1093964735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8022020 con 0x55fd2f650990 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f7104cb0640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 0x7f70e80a4cc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f70f8023000 tx=0x7f70f8021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f70e67fc640 1 -- 172.21.15.96:0/1093964735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8013170 con 0x55fd2f650990 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f70e67fc640 1 -- 172.21.15.96:0/1093964735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8010850 con 0x55fd2f650990 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 -- 172.21.15.96:0/1093964735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 msgr2=0x55fd2f650db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f70f8005c40 tx=0x7f70f800f3c0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 -- 172.21.15.96:0/1093964735 shutdown_connections 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f658980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f651380 0x55fd2f655e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 --2- 172.21.15.96:0/1093964735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f650990 0x55fd2f650db0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 -- 172.21.15.96:0/1093964735 >> 172.21.15.96:0/1093964735 conn(0x55fd2f618c50 msgr2=0x55fd2f64c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 -- 172.21.15.96:0/1093964735 shutdown_connections 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.563+0000 7f71093d0740 1 -- 172.21.15.96:0/1093964735 wait complete. 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 -- start start 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 Processor -- start 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f65aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f662450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f6594a0 con 0x55fd2f651380 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f5710d0 con 0x55fd2f6564f0 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd2f659620 con 0x55fd2f650990 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f71093d0740 10 client.0 _ll_register_callbacks cb 0x55fd2f53a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70fffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f65aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70fffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f65aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:58668/0 (socket says 172.21.15.96:58668) 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43728/0 (socket says 172.21.15.96:43728) 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70fffff640 1 -- 172.21.15.96:0/1558212658 learned_addr learned my addr 172.21.15.96:0/1558212658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f7104cb0640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f662450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 -- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 msgr2=0x55fd2f65aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f65aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 -- 172.21.15.96:0/1558212658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 msgr2=0x55fd2f662450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f662450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.564+0000 7f70ff7fe640 1 -- 172.21.15.96:0/1558212658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2f3f1f70 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f7104cb0640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f662450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31903e5323973321 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4005d50 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f70f4005230 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f8013170 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f70f404b030 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f71093d0740 10 client.16812 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd2f289170 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f8013170 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f70f40039d0 con 0x55fd2f651380 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f71093d0740 10 client.16812 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:01.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.565+0000 7f71093d0740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f71093d0740 20 client.16812 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6669f0 0x55fd2f668e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f71093d0740 4 client.16812 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd2f5c2610 con 0x55fd2f6669f0 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f70fffff640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6669f0 0x55fd2f668e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f70fffff640 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6669f0 0x55fd2f668e30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa2b90c09bc5c39 server_cookie=36d6a9d7b2f75bfd in_seq=0 out_seq=0 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.566+0000 7f70df7fe640 10 client.16812 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f70df7fe640 1 -- 172.21.15.96:0/1558212658 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fd2f5c2610 con 0x55fd2f6669f0 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f70df7fe640 10 client.16812 handle_command_reply: tid=0 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 client.16812 shutdown 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 msgr2=0x55fd2f65b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009230 tx=0x7f70f4004270 comp rx=0 tx=0).stop 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 shutdown_connections 2024-03-18T15:45:01.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f6564f0 0x55fd2f662450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6669f0 0x55fd2f668e30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f65b410 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 --2- 172.21.15.96:0/1558212658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f65aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f70df7fe640 0 client.16812 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.567+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 >> 172.21.15.96:0/1558212658 conn(0x55fd2f618c50 msgr2=0x55fd2f64b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.568+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 shutdown_connections 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.568+0000 7f71093d0740 1 -- 172.21.15.96:0/1558212658 wait complete. 2024-03-18T15:45:01.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.568+0000 7f71093d0740 20 client.16812 trim_cache size 0 max 16384 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 Processor -- start 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 -- start start 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f64f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f651380 0x55fd2f64fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71001a8d20 con 0x55fd2f6564f0 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71001aa920 con 0x55fd2f651380 2024-03-18T15:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.585+0000 7f71093d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71001a93f0 con 0x55fd2f650990 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43730/0 (socket says 172.21.15.96:43730) 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 -- 172.21.15.96:0/4215160775 learned_addr learned my addr 172.21.15.96:0/4215160775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f70fffff640 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f64f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f651380 0x55fd2f64fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 -- 172.21.15.96:0/4215160775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 msgr2=0x55fd2f64f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f64f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 -- 172.21.15.96:0/4215160775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f651380 msgr2=0x55fd2f64fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f651380 0x55fd2f64fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 -- 172.21.15.96:0/4215160775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2f65c050 con 0x55fd2f6564f0 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f7104cb0640 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55fd2f651be0 tx=0x7f70f8003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ad05550fc0bda0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f70e5ffb640 1 -- 172.21.15.96:0/4215160775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8022070 con 0x55fd2f6564f0 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f70e5ffb640 1 -- 172.21.15.96:0/4215160775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f800fd90 con 0x55fd2f6564f0 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.586+0000 7f70e5ffb640 1 -- 172.21.15.96:0/4215160775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8005050 con 0x55fd2f6564f0 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- 172.21.15.96:0/4215160775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 msgr2=0x55fd2f66d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x55fd2f651be0 tx=0x7f70f8003f30 comp rx=0 tx=0).stop 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- 172.21.15.96:0/4215160775 shutdown_connections 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f6564f0 0x55fd2f66d6f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f651380 0x55fd2f64fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 --2- 172.21.15.96:0/4215160775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f650990 0x55fd2f64f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- 172.21.15.96:0/4215160775 >> 172.21.15.96:0/4215160775 conn(0x55fd2f618c50 msgr2=0x55fd2f652880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- 172.21.15.96:0/4215160775 shutdown_connections 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- 172.21.15.96:0/4215160775 wait complete. 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 -- start start 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.587+0000 7f71093d0740 1 Processor -- start 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f650990 0x55fd2f6c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f6564f0 0x55fd2f6d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71001aa250 con 0x55fd2f651380 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71001a9b80 con 0x55fd2f650990 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71001a6d00 con 0x55fd2f6564f0 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f71093d0740 10 client.0 _ll_register_callbacks cb 0x7f70f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43744/0 (socket says 172.21.15.96:43744) 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 -- 172.21.15.96:0/876293172 learned_addr learned my addr 172.21.15.96:0/876293172 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70fffff640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f650990 0x55fd2f6c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 -- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f6564f0 msgr2=0x55fd2f6d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f6564f0 0x55fd2f6d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 -- 172.21.15.96:0/876293172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f650990 msgr2=0x55fd2f6c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f650990 0x55fd2f6c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 -- 172.21.15.96:0/876293172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2f6d0a30 con 0x55fd2f651380 2024-03-18T15:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.588+0000 7f70ff7fe640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70f4003e40 tx=0x7f70f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a18652606d00b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f405c060 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f70f40035f0 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f800fd90 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f70f4003830 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 10 client.16818 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fd2f6d1e10 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f800fd90 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f70f4046450 con 0x55fd2f651380 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 10 client.16818 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 20 client.16818 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6d27e0 0x55fd2f6d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 4 client.16818 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:45:01.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.589+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd2f5bc960 con 0x55fd2f6d27e0 2024-03-18T15:45:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.590+0000 7f70fffff640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6d27e0 0x55fd2f6d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.590+0000 7f70df7fe640 10 client.16818 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.590+0000 7f70fffff640 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6d27e0 0x55fd2f6d4c00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2904bdf74c5c46 server_cookie=69a275452af9e98b in_seq=0 out_seq=0 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f70df7fe640 1 -- 172.21.15.96:0/876293172 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd2f5bc960 con 0x55fd2f6d27e0 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f70df7fe640 10 client.16818 handle_command_reply: tid=0 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 client.16818 shutdown 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 msgr2=0x55fd2f6c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70f4003e40 tx=0x7f70f402bc30 comp rx=0 tx=0).stop 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 shutdown_connections 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd2f6564f0 0x55fd2f6d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fd2f6d27e0 0x55fd2f6d4c00 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fd2f651380 0x55fd2f6c9470 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f70df7fe640 0 client.16818 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 --2- 172.21.15.96:0/876293172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2f650990 0x55fd2f6c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.591+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 >> 172.21.15.96:0/876293172 conn(0x55fd2f618c50 msgr2=0x55fd2f64f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 shutdown_connections 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f71093d0740 1 -- 172.21.15.96:0/876293172 wait complete. 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f71093d0740 20 client.16818 trim_cache size 0 max 16384 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:45:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "f8e45139-760e-4c72-8b48-5597834c9d02", 2024-03-18T15:45:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:45:01.599 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 msgr2=0x7f70e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 0x7f70e80a4cc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f70f8023000 tx=0x7f70f8021000 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 msgr2=0x7f7100197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9c0 tx=0x7f70f000ee80 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 shutdown_connections 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f70e80a2800 0x7f70e80a4cc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7100102290 0x7f710019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71001018d0 0x7f7100198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 --2- 172.21.15.96:0/2460396098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71001006e0 0x7f7100197b10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 >> 172.21.15.96:0/2460396098 conn(0x7f71000fbc70 msgr2=0x7f71000fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 shutdown_connections 2024-03-18T15:45:01.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.592+0000 7f7106713640 1 -- 172.21.15.96:0/2460396098 wait complete. 2024-03-18T15:45:01.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:45:01.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:45:01.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 -- 172.21.15.96:0/2880938811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b501022d0 msgr2=0x7f7b50108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 --2- 172.21.15.96:0/2880938811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b501022d0 0x7f7b50108b60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009ca0 tx=0x7f7b4802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 -- 172.21.15.96:0/2880938811 shutdown_connections 2024-03-18T15:45:01.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 --2- 172.21.15.96:0/2880938811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b501022d0 0x7f7b50108b60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 --2- 172.21.15.96:0/2880938811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b50101910 0x7f7b50101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 --2- 172.21.15.96:0/2880938811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50100720 0x7f7b50100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 -- 172.21.15.96:0/2880938811 >> 172.21.15.96:0/2880938811 conn(0x7f7b500fbc90 msgr2=0x7f7b500fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 -- 172.21.15.96:0/2880938811 shutdown_connections 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.716+0000 7f7b551d3640 1 -- 172.21.15.96:0/2880938811 wait complete. 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 Processor -- start 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 -- start start 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50101910 0x7f7b50198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b501022d0 0x7f7b5019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010c380 con 0x7f7b50100720 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010c200 con 0x7f7b50101910 2024-03-18T15:45:01.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b551d3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5010c500 con 0x7f7b501022d0 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43758/0 (socket says 172.21.15.96:43758) 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/473721327 learned_addr learned my addr 172.21.15.96:0/473721327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b47fff640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50101910 0x7f7b50198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b501022d0 0x7f7b5019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/473721327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b501022d0 msgr2=0x7f7b5019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b501022d0 0x7f7b5019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/473721327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50101910 msgr2=0x7f7b50198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.717+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50101910 0x7f7b50198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/473721327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b5019f730 con 0x7f7b50100720 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000e9c0 tx=0x7f7b4000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b4cd99640 1 -- 172.21.15.96:0/473721327 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000cd80 con 0x7f7b50100720 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5019fa20 con 0x7f7b50100720 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b4cd99640 1 -- 172.21.15.96:0/473721327 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b400035f0 con 0x7f7b50100720 2024-03-18T15:45:01.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b4cd99640 1 -- 172.21.15.96:0/473721327 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40010690 con 0x7f7b50100720 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.718+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b5019ffc0 con 0x7f7b50100720 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.720+0000 7f7b4cd99640 1 -- 172.21.15.96:0/473721327 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7b400037a0 con 0x7f7b50100720 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.720+0000 7f7b57db7740 1 Processor -- start 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.720+0000 7f7b57db7740 1 -- start start 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4cd99640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 0x7f7b3c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d0b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583d4c68500 con 0x5583d4d08cd0 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583d4c10be0 con 0x5583d4d03170 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b57db7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583d4c645a0 con 0x5583d4d03b60 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 0x7f7b3c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d0b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:38884/0 (socket says 172.21.15.96:38884) 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/3527608976 learned_addr learned my addr 172.21.15.96:0/3527608976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b47fff640 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4cd99640 1 -- 172.21.15.96:0/473721327 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b400cee40 con 0x7f7b50100720 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/3527608976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 msgr2=0x5583d4d08660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d08660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/3527608976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 msgr2=0x5583d4d0b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d0b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4ed9d640 1 -- 172.21.15.96:0/3527608976 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d4c244b0 con 0x5583d4d03170 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.721+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d0b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 0x7f7b3c0a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7b48006010 tx=0x7f7b4802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b400058c0 tx=0x7f7b400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48538fc62abb3205 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b457fa640 1 -- 172.21.15.96:0/3527608976 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4009c8e0 con 0x5583d4d03170 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b457fa640 1 -- 172.21.15.96:0/3527608976 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b400083e0 con 0x5583d4d03170 2024-03-18T15:45:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b457fa640 1 -- 172.21.15.96:0/3527608976 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40008220 con 0x5583d4d03170 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b57db7740 1 -- 172.21.15.96:0/3527608976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 msgr2=0x5583d4d03590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.722+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b400058c0 tx=0x7f7b400046c0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- 172.21.15.96:0/3527608976 shutdown_connections 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d0b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d08660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3527608976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4d03590 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- 172.21.15.96:0/3527608976 >> 172.21.15.96:0/3527608976 conn(0x5583d4cfe6e0 msgr2=0x5583d4cfeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- 172.21.15.96:0/3527608976 shutdown_connections 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- 172.21.15.96:0/3527608976 wait complete. 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- start start 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 Processor -- start 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d0d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d0db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583d4d0bc80 con 0x5583d4d08cd0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583d4c24160 con 0x5583d4d03b60 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583d4d0be00 con 0x5583d4d03170 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b57db7740 10 client.0 _ll_register_callbacks cb 0x5583d4c08150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d0d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43780/0 (socket says 172.21.15.96:43780) 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b47fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d0db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.723+0000 7f7b4f59e640 1 -- 172.21.15.96:0/3434543582 learned_addr learned my addr 172.21.15.96:0/3434543582 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 -- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 msgr2=0x5583d4d0d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d0d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 -- 172.21.15.96:0/3434543582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 msgr2=0x5583d4d0db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d0db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 -- 172.21.15.96:0/3434543582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d4a467d0 con 0x5583d4d08cd0 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d0d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:01.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009ca0 tx=0x7f7b4802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1f859c627d6be4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48039040 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b48004250 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b400083e0 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b48032340 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 10 client.16830 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583d497c3f0 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b400083e0 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7b48034800 con 0x5583d4d08cd0 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 10 client.16830 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 20 client.16830 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d19100 0x5583d4d1b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 4 client.16830 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:01.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.724+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d4c75a40 con 0x5583d4d19100 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.725+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d19100 0x5583d4d1b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.725+0000 7f7b367fc640 10 client.16830 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.725+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d19100 0x5583d4d1b5a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f84029fa9acde2 server_cookie=676f60776490bb63 in_seq=0 out_seq=0 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b367fc640 1 -- 172.21.15.96:0/3434543582 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5583d4c75a40 con 0x5583d4d19100 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b367fc640 10 client.16830 handle_command_reply: tid=0 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 client.16830 shutdown 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 msgr2=0x5583d4d14ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009ca0 tx=0x7f7b4802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 shutdown_connections 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d19100 0x5583d4d1b5a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d14ba0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b367fc640 0 client.16830 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d0db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 --2- 172.21.15.96:0/3434543582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d0d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 >> 172.21.15.96:0/3434543582 conn(0x5583d4cfe6e0 msgr2=0x5583d4cfd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 shutdown_connections 2024-03-18T15:45:01.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.726+0000 7f7b57db7740 1 -- 172.21.15.96:0/3434543582 wait complete. 2024-03-18T15:45:01.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.727+0000 7f7b57db7740 20 client.16830 trim_cache size 0 max 16384 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 Processor -- start 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 -- start start 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4cf98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a8dd0 con 0x5583d4d08cd0 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501aa9d0 con 0x5583d4d03170 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b57db7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a94a0 con 0x5583d4d03b60 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43792/0 (socket says 172.21.15.96:43792) 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b4f59e640 1 -- 172.21.15.96:0/4214066280 learned_addr learned my addr 172.21.15.96:0/4214066280 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.744+0000 7f7b47fff640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4cf98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 -- 172.21.15.96:0/4214066280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 msgr2=0x5583d4d14b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d14b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 -- 172.21.15.96:0/4214066280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 msgr2=0x5583d4cf98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4cf98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 -- 172.21.15.96:0/4214066280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d493c0f0 con 0x5583d4d08cd0 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b47fff640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:01.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802c630 tx=0x7f7b48009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9ad15c47e440da server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b44ff9640 1 -- 172.21.15.96:0/4214066280 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4805d070 con 0x5583d4d08cd0 2024-03-18T15:45:01.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b44ff9640 1 -- 172.21.15.96:0/4214066280 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b48032340 con 0x5583d4d08cd0 2024-03-18T15:45:01.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.745+0000 7f7b44ff9640 1 -- 172.21.15.96:0/4214066280 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b48004310 con 0x5583d4d08cd0 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- 172.21.15.96:0/4214066280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 msgr2=0x5583d4cfd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7b4802c630 tx=0x7f7b48009ba0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- 172.21.15.96:0/4214066280 shutdown_connections 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4cfd070 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03b60 0x5583d4d14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- 172.21.15.96:0/4214066280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03170 0x5583d4cf98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- 172.21.15.96:0/4214066280 >> 172.21.15.96:0/4214066280 conn(0x5583d4cfe6e0 msgr2=0x5583d4cfeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- 172.21.15.96:0/4214066280 shutdown_connections 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- 172.21.15.96:0/4214066280 wait complete. 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- start start 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 Processor -- start 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d12060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501aa300 con 0x5583d4d08cd0 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a9c30 con 0x5583d4d03b60 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b501a6e00 con 0x5583d4d03170 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.746+0000 7f7b57db7740 10 client.0 _ll_register_callbacks cb 0x7f7b480329a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43806/0 (socket says 172.21.15.96:43806) 2024-03-18T15:45:01.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d12060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 -- 172.21.15.96:0/393765273 learned_addr learned my addr 172.21.15.96:0/393765273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b47fff640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 -- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 msgr2=0x5583d4d12060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d12060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 -- 172.21.15.96:0/393765273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 msgr2=0x5583d4d125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 -- 172.21.15.96:0/393765273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d4c2f9d0 con 0x5583d4d08cd0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b4f59e640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7b48032bc0 tx=0x7f7b48009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f3ebde32361aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b367fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4805d050 con 0x5583d4d08cd0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7b480346d0 con 0x5583d4d08cd0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b48032340 con 0x5583d4d08cd0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7b48048040 con 0x5583d4d08cd0 2024-03-18T15:45:01.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b57db7740 10 client.16836 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5583d4d14ce0 con 0x5583d4d08cd0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.747+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b48032340 con 0x5583d4d08cd0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7b48005300 con 0x5583d4d08cd0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 10 client.16836 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 20 client.16836 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d7f8d0 0x5583d4d81cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 4 client.16836 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583d4c6fd90 con 0x5583d4d7f8d0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d7f8d0 0x5583d4d81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b367fc640 10 client.16836 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b4ed9d640 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d7f8d0 0x5583d4d81cc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78df5273b7c0f5c3 server_cookie=c798b4a8d699cf6e in_seq=0 out_seq=0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b367fc640 1 -- 172.21.15.96:0/393765273 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5583d4c6fd90 con 0x5583d4d7f8d0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b367fc640 10 client.16836 handle_command_reply: tid=0 2024-03-18T15:45:01.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.748+0000 7f7b57db7740 1 client.16836 shutdown 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 msgr2=0x5583d4d135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7b48032bc0 tx=0x7f7b48009560 comp rx=0 tx=0).stop 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 shutdown_connections 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5583d4d7f8d0 0x5583d4d81cc0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583d4d08cd0 0x5583d4d135b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583d4d03b60 0x5583d4d125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b367fc640 0 client.16836 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 --2- 172.21.15.96:0/393765273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583d4d03170 0x5583d4d12060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 >> 172.21.15.96:0/393765273 conn(0x5583d4cfe6e0 msgr2=0x5583d4cfeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 shutdown_connections 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 1 -- 172.21.15.96:0/393765273 wait complete. 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b57db7740 20 client.16836 trim_cache size 0 max 16384 2024-03-18T15:45:01.756 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "f8e45139-760e-4c72-8b48-5597834c9d02": { 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "f8e45139-760e-4c72-8b48-5597834c9d02", 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:45:01.757 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 msgr2=0x7f7b3c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 0x7f7b3c0a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7b48006010 tx=0x7f7b4802e040 comp rx=0 tx=0).stop 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 msgr2=0x7f7b50197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000e9c0 tx=0x7f7b4000ee80 comp rx=0 tx=0).stop 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 shutdown_connections 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7b3c0a2800 0x7f7b3c0a4cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b501022d0 0x7f7b5019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b50101910 0x7f7b50198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 --2- 172.21.15.96:0/473721327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b50100720 0x7f7b50197b10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:01.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 >> 172.21.15.96:0/473721327 conn(0x7f7b500fbc90 msgr2=0x7f7b500fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:01.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 shutdown_connections 2024-03-18T15:45:01.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:01.749+0000 7f7b551d3640 1 -- 172.21.15.96:0/473721327 wait complete. 2024-03-18T15:45:01.761 INFO:tasks.cephfs.filesystem:scrub status for tag:f8e45139-760e-4c72-8b48-5597834c9d02 - {'path': '/', 'tag': 'f8e45139-760e-4c72-8b48-5597834c9d02', 'options': 'recursive,force'} 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: pgmap v500: 97 pgs: 97 active+clean; 636 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:02 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: pgmap v500: 97 pgs: 97 active+clean; 636 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:02.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:02 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: pgmap v500: 97 pgs: 97 active+clean; 636 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 489 op/s 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:02 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:03.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:45:03.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:03 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:45:03.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:03 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:04.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:45:04.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:03 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:45:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:04 smithi193 ceph-mon[36702]: pgmap v501: 97 pgs: 97 active+clean; 641 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-18T15:45:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:04 smithi096 ceph-mon[31930]: pgmap v501: 97 pgs: 97 active+clean; 641 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-18T15:45:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:04 smithi137 ceph-mon[35868]: pgmap v501: 97 pgs: 97 active+clean; 641 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 533 op/s 2024-03-18T15:45:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:06 smithi193 ceph-mon[36702]: pgmap v502: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-03-18T15:45:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:06 smithi096 ceph-mon[31930]: pgmap v502: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-03-18T15:45:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:06 smithi137 ceph-mon[35868]: pgmap v502: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 489 op/s 2024-03-18T15:45:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:08 smithi193 ceph-mon[36702]: pgmap v503: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-03-18T15:45:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:08 smithi096 ceph-mon[31930]: pgmap v503: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-03-18T15:45:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:08 smithi137 ceph-mon[35868]: pgmap v503: 97 pgs: 97 active+clean; 644 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 4.5 MiB/s wr, 444 op/s 2024-03-18T15:45:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.661+0000 7ff5d6ffd640 1 -- 172.21.15.96:0/1108314697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc004e60 con 0x7ff5d8073510 2024-03-18T15:45:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.661+0000 7ff5de577640 1 -- 172.21.15.96:0/1108314697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 msgr2=0x7ff5d8073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.662 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.661+0000 7ff5de577640 1 --2- 172.21.15.96:0/1108314697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d8073930 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009d50 tx=0x7ff5cc02c080 comp rx=0 tx=0).stop 2024-03-18T15:45:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.666+0000 7ff5de577640 1 -- 172.21.15.96:0/1108314697 shutdown_connections 2024-03-18T15:45:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.666+0000 7ff5de577640 1 --2- 172.21.15.96:0/1108314697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8078da0 0x7ff5d807b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.666+0000 7ff5de577640 1 --2- 172.21.15.96:0/1108314697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d80763f0 0x7ff5d8078860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.666+0000 7ff5de577640 1 --2- 172.21.15.96:0/1108314697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d8073930 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.666+0000 7ff5de577640 1 -- 172.21.15.96:0/1108314697 >> 172.21.15.96:0/1108314697 conn(0x7ff5d806e650 msgr2=0x7ff5d8070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.667+0000 7ff5de577640 1 -- 172.21.15.96:0/1108314697 shutdown_connections 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.667+0000 7ff5de577640 1 -- 172.21.15.96:0/1108314697 wait complete. 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.667+0000 7ff5de577640 1 Processor -- start 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 -- start start 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80763f0 0x7ff5d81b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d8078da0 0x7ff5d81b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d807ec30 con 0x7ff5d8073510 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d807eab0 con 0x7ff5d80763f0 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5de577640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d807edb0 con 0x7ff5d8078da0 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.668+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59074/0 (socket says 172.21.15.96:59074) 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 -- 172.21.15.96:0/4074434560 learned_addr learned my addr 172.21.15.96:0/4074434560 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 -- 172.21.15.96:0/4074434560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d8078da0 msgr2=0x7ff5d81b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d8078da0 0x7ff5d81b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 -- 172.21.15.96:0/4074434560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80763f0 msgr2=0x7ff5d81b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80763f0 0x7ff5d81b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 -- 172.21.15.96:0/4074434560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d81b7c80 con 0x7ff5d8073510 2024-03-18T15:45:10.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.669+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc000c00 tx=0x7ff5cc0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.696+0000 7ff5d57fa640 1 -- 172.21.15.96:0/4074434560 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0042c0 con 0x7ff5d8073510 2024-03-18T15:45:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.696+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d807fe50 con 0x7ff5d8073510 2024-03-18T15:45:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.696+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d80803a0 con 0x7ff5d8073510 2024-03-18T15:45:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.698+0000 7ff5d57fa640 1 -- 172.21.15.96:0/4074434560 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc004460 con 0x7ff5d8073510 2024-03-18T15:45:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.698+0000 7ff5d57fa640 1 -- 172.21.15.96:0/4074434560 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0035f0 con 0x7ff5d8073510 2024-03-18T15:45:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.700+0000 7ff5e1166740 1 Processor -- start 2024-03-18T15:45:10.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.700+0000 7ff5e1166740 1 -- start start 2024-03-18T15:45:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5d57fa640 1 -- 172.21.15.96:0/4074434560 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff5cc003790 con 0x7ff5d8073510 2024-03-18T15:45:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 0x56325013e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 0x5632501439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 0x5632501464f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5e1166740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632500a3a40 con 0x56325013eef0 2024-03-18T15:45:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.701+0000 7ff5e1166740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56325004c920 con 0x563250144060 2024-03-18T15:45:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5e1166740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56325009fae0 con 0x56325013e500 2024-03-18T15:45:10.703 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5dcb14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 0x56325013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5dcb14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 0x56325013e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:40358/0 (socket says 172.21.15.96:40358) 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5dcb14640 1 -- 172.21.15.96:0/2858421879 learned_addr learned my addr 172.21.15.96:0/2858421879 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 0x5632501464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.702+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 0x5632501439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d77fe640 1 -- 172.21.15.96:0/2858421879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 msgr2=0x56325013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 0x56325013e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d77fe640 1 -- 172.21.15.96:0/2858421879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 msgr2=0x5632501439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 0x5632501439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d77fe640 1 -- 172.21.15.96:0/2858421879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563250064460 con 0x563250144060 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.710+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 0x5632501439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.711+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 0x5632501464f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009b10 tx=0x7ff5c800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cb923d18b13a7b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.711+0000 7ff5d57fa640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 0x7ff5c00a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.711+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 0x7ff5c00a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.712+0000 7ff5d57fa640 1 -- 172.21.15.96:0/4074434560 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff5cc0f9360 con 0x7ff5d8073510 2024-03-18T15:45:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.712+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 0x7ff5c00a4ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56325013f750 tx=0x7ff5d00099b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.730+0000 7ff5be7fc640 1 -- 172.21.15.96:0/2858421879 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800ed50 con 0x563250144060 2024-03-18T15:45:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.731+0000 7ff5be7fc640 1 -- 172.21.15.96:0/2858421879 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8005020 con 0x563250144060 2024-03-18T15:45:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.731+0000 7ff5be7fc640 1 -- 172.21.15.96:0/2858421879 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8004c40 con 0x563250144060 2024-03-18T15:45:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.732+0000 7ff5e1166740 1 -- 172.21.15.96:0/2858421879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 msgr2=0x5632501464f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.732 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.732+0000 7ff5e1166740 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 0x5632501464f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009b10 tx=0x7ff5c800e350 comp rx=0 tx=0).stop 2024-03-18T15:45:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.735+0000 7ff5e1166740 1 -- 172.21.15.96:0/2858421879 shutdown_connections 2024-03-18T15:45:10.735 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.735+0000 7ff5e1166740 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250144060 0x5632501464f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.735+0000 7ff5e1166740 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013eef0 0x5632501439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.735+0000 7ff5e1166740 1 --2- 172.21.15.96:0/2858421879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013e500 0x56325013e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.735+0000 7ff5e1166740 1 -- 172.21.15.96:0/2858421879 >> 172.21.15.96:0/2858421879 conn(0x563250139a70 msgr2=0x563250139e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:10.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.739+0000 7ff5e1166740 1 -- 172.21.15.96:0/2858421879 shutdown_connections 2024-03-18T15:45:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.739+0000 7ff5e1166740 1 -- 172.21.15.96:0/2858421879 wait complete. 2024-03-18T15:45:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.739+0000 7ff5e1166740 1 -- start start 2024-03-18T15:45:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 Processor -- start 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013e500 0x563250159840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 0x563250160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563250147010 con 0x563250144060 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56325005fd70 con 0x56325013e500 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563250147190 con 0x56325013eef0 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5e1166740 10 client.0 _ll_register_callbacks cb 0x56324fdb7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.740+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5dcb14640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 0x563250160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:40366/0 (socket says 172.21.15.96:40366) 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 -- 172.21.15.96:0/681816075 learned_addr learned my addr 172.21.15.96:0/681816075 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:10.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013e500 0x563250159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 -- 172.21.15.96:0/681816075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013e500 msgr2=0x563250159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013e500 0x563250159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 -- 172.21.15.96:0/681816075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 msgr2=0x563250160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 0x563250160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5d77fe640 1 -- 172.21.15.96:0/681816075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56324fdb8210 con 0x56325013eef0 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.741+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 0x563250160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:10.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.742+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c8005400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309beae35645056c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.759+0000 7ff5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:45:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.759+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8029060 con 0x56325013eef0 2024-03-18T15:45:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.759+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff5c8004b50 con 0x56325013eef0 2024-03-18T15:45:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.759+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c8005020 con 0x56325013eef0 2024-03-18T15:45:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.762+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5c8004d50 con 0x56325013eef0 2024-03-18T15:45:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.762+0000 7ff5e1166740 10 client.26689 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.762+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56324fe84620 con 0x56325013eef0 2024-03-18T15:45:10.763 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8005020 con 0x56325013eef0 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff5c8010430 con 0x56325013eef0 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 10 client.26689 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 20 client.26689 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56325014aff0 0x56325014d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 4 client.26689 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632500b0f80 con 0x56325014aff0 2024-03-18T15:45:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.763+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56325014aff0 0x56325014d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.764+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56325014aff0 0x56325014d450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88658c34641bbc21 server_cookie=5a3fed314cdc47fb in_seq=0 out_seq=0 2024-03-18T15:45:10.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.767+0000 7ff5b77fe640 10 client.26689 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.767+0000 7ff5b77fe640 1 -- 172.21.15.96:0/681816075 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5632500b0f80 con 0x56325014aff0 2024-03-18T15:45:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.767+0000 7ff5b77fe640 10 client.26689 handle_command_reply: tid=0 2024-03-18T15:45:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.767+0000 7ff5e1166740 1 client.26689 shutdown 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 msgr2=0x563250159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8000c00 tx=0x7ff5c8005400 comp rx=0 tx=0).stop 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 shutdown_connections 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56325014aff0 0x56325014d450 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563250144060 0x563250160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x563250159d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 --2- 172.21.15.96:0/681816075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013e500 0x563250159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.786 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.785+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 >> 172.21.15.96:0/681816075 conn(0x563250139a70 msgr2=0x563250138c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.789+0000 7ff5b77fe640 0 client.26689 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.789+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 shutdown_connections 2024-03-18T15:45:10.789 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.789+0000 7ff5e1166740 1 -- 172.21.15.96:0/681816075 wait complete. 2024-03-18T15:45:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.791+0000 7ff5e1166740 20 client.26689 trim_cache size 0 max 16384 2024-03-18T15:45:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.896+0000 7ff5e1166740 1 Processor -- start 2024-03-18T15:45:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.896+0000 7ff5e1166740 1 -- start start 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.896+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x563250160d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013eef0 0x56325013d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5e1166740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400e650 con 0x56325013e500 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5e1166740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c4010250 con 0x56325013eef0 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5e1166740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400ed20 con 0x563250139a70 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x563250160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:40380/0 (socket says 172.21.15.96:40380) 2024-03-18T15:45:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 -- 172.21.15.96:0/1083094011 learned_addr learned my addr 172.21.15.96:0/1083094011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013eef0 0x56325013d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 -- 172.21.15.96:0/1083094011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013eef0 msgr2=0x56325013d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013eef0 0x56325013d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.897+0000 7ff5d7fff640 1 -- 172.21.15.96:0/1083094011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 msgr2=0x563250160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.898+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x563250160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.898+0000 7ff5d7fff640 1 -- 172.21.15.96:0/1083094011 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325013f2d0 con 0x563250139a70 2024-03-18T15:45:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.898+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x563250160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.898+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc006010 tx=0x7ff5cc02c080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8feac04cba451ac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:10 smithi193 ceph-mon[36702]: pgmap v504: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-18T15:45:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:10 smithi096 ceph-mon[31930]: pgmap v504: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-18T15:45:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.921+0000 7ff5bdffb640 1 -- 172.21.15.96:0/1083094011 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0449c0 con 0x563250139a70 2024-03-18T15:45:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.935+0000 7ff5bdffb640 1 -- 172.21.15.96:0/1083094011 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc0c61d0 con 0x563250139a70 2024-03-18T15:45:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.936+0000 7ff5bdffb640 1 -- 172.21.15.96:0/1083094011 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc0c3c00 con 0x563250139a70 2024-03-18T15:45:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 -- 172.21.15.96:0/1083094011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 msgr2=0x56325013ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc006010 tx=0x7ff5cc02c080 comp rx=0 tx=0).stop 2024-03-18T15:45:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 -- 172.21.15.96:0/1083094011 shutdown_connections 2024-03-18T15:45:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56325013eef0 0x56325013d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x563250160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1083094011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563250139a70 0x56325013ccd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.937+0000 7ff5e1166740 1 -- 172.21.15.96:0/1083094011 >> 172.21.15.96:0/1083094011 conn(0x563250143b60 msgr2=0x5632501401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.938+0000 7ff5e1166740 1 -- 172.21.15.96:0/1083094011 shutdown_connections 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.938+0000 7ff5e1166740 1 -- 172.21.15.96:0/1083094011 wait complete. 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.938+0000 7ff5e1166740 1 -- start start 2024-03-18T15:45:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.939+0000 7ff5e1166740 1 Processor -- start 2024-03-18T15:45:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.939+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 0x563250160c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.939+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x56325015d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5e1166740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x56325015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5e1166740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400fb80 con 0x56325013e500 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x56325015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5e1166740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400f4b0 con 0x563250139a70 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5e1166740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c400c630 con 0x56325013eef0 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 0x563250160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5e1166740 10 client.0 _ll_register_callbacks cb 0x7ff5d0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 0x563250160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:56816/0 (socket says 172.21.15.96:56816) 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5d7fff640 1 -- 172.21.15.96:0/1369998405 learned_addr learned my addr 172.21.15.96:0/1369998405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.940+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x56325015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 -- 172.21.15.96:0/1369998405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 msgr2=0x563250160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 0x563250160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 -- 172.21.15.96:0/1369998405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 msgr2=0x56325015d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x56325015d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 -- 172.21.15.96:0/1369998405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632501daaf0 con 0x56325013eef0 2024-03-18T15:45:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5d77fe640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x56325015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.941+0000 7ff5dcb14640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x56325015dcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00049a0 tx=0x7ff5d0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=522d12cdd1a83470 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d002d060 con 0x56325013eef0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff5d0009040 con 0x56325013eef0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5cc0c61d0 con 0x56325013eef0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff5d000fea0 con 0x56325013eef0 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5e1166740 10 client.26695 fetch_fsmap learned FSMap version 33 2024-03-18T15:45:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.992+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5632501dbe30 con 0x56325013eef0 2024-03-18T15:45:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.993+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5cc0c61d0 con 0x56325013eef0 2024-03-18T15:45:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.994+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff5d001a050 con 0x56325013eef0 2024-03-18T15:45:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 10 client.26695 fetch_fsmap finished waiting for FSMap version 33 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 20 client.26695 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563250144060 0x5632501de7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 4 client.26695 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632500ab2d0 con 0x563250144060 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563250144060 0x5632501de7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5d7fff640 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563250144060 0x5632501de7d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=388d6d8db7f2125b server_cookie=4bf6c8843df2d6ac in_seq=0 out_seq=0 2024-03-18T15:45:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:10.995+0000 7ff5b77fe640 10 client.26695 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:11.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:10 smithi137 ceph-mon[35868]: pgmap v504: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-03-18T15:45:11.331 INFO:tasks.workunit.client.0.smithi096.stdout: 126 1119116 209 809218 275 641209 796 2024-03-18T15:45:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5b77fe640 1 -- 172.21.15.96:0/1369998405 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5632500ab2d0 con 0x563250144060 2024-03-18T15:45:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5b77fe640 10 client.26695 handle_command_reply: tid=0 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 client.26695 shutdown 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 msgr2=0x56325015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x56325015dcc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00049a0 tx=0x7ff5d0004a40 comp rx=0 tx=0).stop 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 shutdown_connections 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56325013eef0 0x56325015dcc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563250144060 0x5632501de7d0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56325013e500 0x56325015d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.418 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 --2- 172.21.15.96:0/1369998405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563250139a70 0x563250160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 >> 172.21.15.96:0/1369998405 conn(0x563250143b60 msgr2=0x5632501597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.416+0000 7ff5b77fe640 0 client.26695 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.417+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 shutdown_connections 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.417+0000 7ff5e1166740 1 -- 172.21.15.96:0/1369998405 wait complete. 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.417+0000 7ff5e1166740 20 client.26695 trim_cache size 0 max 16384 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:45:11.419 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:45:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.425+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 msgr2=0x7ff5c00a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.425+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 0x7ff5c00a4ec0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x56325013f750 tx=0x7ff5d00099b0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.426+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 msgr2=0x7ff5d81b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:11.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.427+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc000c00 tx=0x7ff5cc0049c0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.429+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 shutdown_connections 2024-03-18T15:45:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.429+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5c00a2a00 0x7ff5c00a4ec0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.430+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5d8078da0 0x7ff5d81b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.430+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d80763f0 0x7ff5d81b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.430+0000 7ff5de577640 1 --2- 172.21.15.96:0/4074434560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5d8073510 0x7ff5d81b0060 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.430+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 >> 172.21.15.96:0/4074434560 conn(0x7ff5d806e650 msgr2=0x7ff5d8077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:11.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.432+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 shutdown_connections 2024-03-18T15:45:11.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:11.432+0000 7ff5de577640 1 -- 172.21.15.96:0/4074434560 wait complete. 2024-03-18T15:45:12.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:12 smithi193 ceph-mon[36702]: pgmap v505: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-03-18T15:45:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:12 smithi096 ceph-mon[31930]: pgmap v505: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-03-18T15:45:13.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:12 smithi137 ceph-mon[35868]: pgmap v505: 97 pgs: 97 active+clean; 647 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 1.9 MiB/s wr, 205 op/s 2024-03-18T15:45:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:14 smithi193 ceph-mon[36702]: pgmap v506: 97 pgs: 97 active+clean; 649 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-18T15:45:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:14 smithi096 ceph-mon[31930]: pgmap v506: 97 pgs: 97 active+clean; 649 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-18T15:45:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:14 smithi137 ceph-mon[35868]: pgmap v506: 97 pgs: 97 active+clean; 649 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-03-18T15:45:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:15 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:45:16.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:15 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:45:16.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:15 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:45:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:16 smithi096 ceph-mon[31930]: pgmap v507: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-18T15:45:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:16 smithi137 ceph-mon[35868]: pgmap v507: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-18T15:45:17.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:16 smithi193 ceph-mon[36702]: pgmap v507: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-03-18T15:45:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:18 smithi096 ceph-mon[31930]: pgmap v508: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 926 KiB/s wr, 136 op/s 2024-03-18T15:45:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:18 smithi137 ceph-mon[35868]: pgmap v508: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 926 KiB/s wr, 136 op/s 2024-03-18T15:45:19.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:18 smithi193 ceph-mon[36702]: pgmap v508: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 926 KiB/s wr, 136 op/s 2024-03-18T15:45:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:20 smithi096 ceph-mon[31930]: pgmap v509: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-03-18T15:45:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:20 smithi137 ceph-mon[35868]: pgmap v509: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-03-18T15:45:21.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:20 smithi193 ceph-mon[36702]: pgmap v509: 97 pgs: 97 active+clean; 637 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 202 op/s 2024-03-18T15:45:21.331 INFO:tasks.workunit.client.0.smithi096.stdout: 132 1913843 329 1396975 372 1116227 1081 2024-03-18T15:45:22.941 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:22 smithi096 ceph-mon[31930]: pgmap v510: 97 pgs: 97 active+clean; 632 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-03-18T15:45:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:22 smithi137 ceph-mon[35868]: pgmap v510: 97 pgs: 97 active+clean; 632 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-03-18T15:45:23.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:22 smithi193 ceph-mon[36702]: pgmap v510: 97 pgs: 97 active+clean; 632 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 964 KiB/s rd, 1.1 MiB/s wr, 203 op/s 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 -- 172.21.15.96:0/1030321756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5441033e0 msgr2=0x7fa544109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 --2- 172.21.15.96:0/1030321756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5441033e0 0x7fa544109c70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009ca0 tx=0x7fa53c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 -- 172.21.15.96:0/1030321756 shutdown_connections 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 --2- 172.21.15.96:0/1030321756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5441033e0 0x7fa544109c70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 --2- 172.21.15.96:0/1030321756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 0x7fa544102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 --2- 172.21.15.96:0/1030321756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa544101830 0x7fa544101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 -- 172.21.15.96:0/1030321756 >> 172.21.15.96:0/1030321756 conn(0x7fa5440fcda0 msgr2=0x7fa5440ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 -- 172.21.15.96:0/1030321756 shutdown_connections 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 -- 172.21.15.96:0/1030321756 wait complete. 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.671+0000 7fa54a709640 1 Processor -- start 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 -- start start 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 0x7fa544197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5441033e0 0x7fa54419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410d4b0 con 0x7fa544101830 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410d330 con 0x7fa5441033e0 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa54a709640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410d630 con 0x7fa544102a20 2024-03-18T15:45:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42388/0 (socket says 172.21.15.96:42388) 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 -- 172.21.15.96:0/3449899227 learned_addr learned my addr 172.21.15.96:0/3449899227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa5437fe640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 0x7fa544197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5441033e0 0x7fa54419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 -- 172.21.15.96:0/3449899227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 msgr2=0x7fa544197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 0x7fa544197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 -- 172.21.15.96:0/3449899227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5441033e0 msgr2=0x7fa54419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5441033e0 0x7fa54419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 -- 172.21.15.96:0/3449899227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54419f650 con 0x7fa544101830 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa543fff640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa53400e9c0 tx=0x7fa53400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.672+0000 7fa5417fa640 1 -- 172.21.15.96:0/3449899227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53400cd80 con 0x7fa544101830 2024-03-18T15:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.673+0000 7fa5417fa640 1 -- 172.21.15.96:0/3449899227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5340035f0 con 0x7fa544101830 2024-03-18T15:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.673+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54419f940 con 0x7fa544101830 2024-03-18T15:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.673+0000 7fa5417fa640 1 -- 172.21.15.96:0/3449899227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534010690 con 0x7fa544101830 2024-03-18T15:45:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.673+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54419fe90 con 0x7fa544101830 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa5417fa640 1 -- 172.21.15.96:0/3449899227 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fa5340037a0 con 0x7fa544101830 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 Processor -- start 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 -- start start 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa5417fa640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 0x7fa52c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 0x562d1a0ba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d1a017a40 con 0x562d1a0b2340 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d19fc0920 con 0x562d1a0b2d30 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d1a013ae0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.675+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 0x7fa52c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 0x562d1a0ba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:56890/0 (socket says 172.21.15.96:56890) 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42392/0 (socket says 172.21.15.96:42392) 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 -- 172.21.15.96:0/3936802569 learned_addr learned my addr 172.21.15.96:0/3936802569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5417fa640 1 -- 172.21.15.96:0/3449899227 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa5340cee40 con 0x7fa544101830 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 -- 172.21.15.96:0/3936802569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 msgr2=0x562d1a0ba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 0x562d1a0ba330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 -- 172.21.15.96:0/3936802569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 msgr2=0x562d1a0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 -- 172.21.15.96:0/3936802569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d19fd8460 con 0x562d1a0b2d30 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 0x562d1a0ba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa543fff640 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 0x7fa52c0a4d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005e30 tx=0x7fa53c0323b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa5437fe640 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa538009ca0 tx=0x7fa538011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23db7623aae6c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa52a7fc640 1 -- 172.21.15.96:0/3936802569 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53801e070 con 0x562d1a0b2d30 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa52a7fc640 1 -- 172.21.15.96:0/3936802569 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538009e50 con 0x562d1a0b2d30 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.676+0000 7fa52a7fc640 1 -- 172.21.15.96:0/3936802569 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538018770 con 0x562d1a0b2d30 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3936802569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 msgr2=0x562d1a0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa538009ca0 tx=0x7fa538011810 comp rx=0 tx=0).stop 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3936802569 shutdown_connections 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b7ea0 0x562d1a0ba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0b7830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3936802569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3936802569 >> 172.21.15.96:0/3936802569 conn(0x562d1a0ad8b0 msgr2=0x562d1a0adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3936802569 shutdown_connections 2024-03-18T15:45:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3936802569 wait complete. 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.677+0000 7fa54d3d3740 1 -- start start 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 Processor -- start 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0bcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562d1a0bae50 con 0x562d1a0b7ea0 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d19fd3d70 con 0x562d1a0b2d30 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d1a0bafd0 con 0x562d1a0b2340 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 10 client.0 _ll_register_callbacks cb 0x562d19d2ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42406/0 (socket says 172.21.15.96:42406) 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 -- 172.21.15.96:0/501222037 learned_addr learned my addr 172.21.15.96:0/501222037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa5437fe640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0bcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa543fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 -- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 msgr2=0x562d1a0bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 -- 172.21.15.96:0/501222037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 msgr2=0x562d1a0bcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0bcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 -- 172.21.15.96:0/501222037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d19d2c210 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa548ca6640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009ca0 tx=0x7fa53c045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389dc7c39ad58662 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa5237fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c039040 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa53c034b90 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.678+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa538009e50 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa53c034d90 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 10 client.16863 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d19df8620 con 0x562d1a0b7ea0 2024-03-18T15:45:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa538009e50 con 0x562d1a0b7ea0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fa53c057430 con 0x562d1a0b7ea0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 10 client.16863 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 20 client.16863 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a0c84f0 0x562d1a0ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 4 client.16863 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d1a024f80 con 0x562d1a0c84f0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.679+0000 7fa543fff640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a0c84f0 0x562d1a0ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.680+0000 7fa543fff640 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a0c84f0 0x562d1a0ca950 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=742dd21387c0738d server_cookie=5ec839f15b88bc21 in_seq=0 out_seq=0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.680+0000 7fa5237fe640 10 client.16863 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa5237fe640 1 -- 172.21.15.96:0/501222037 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562d1a024f80 con 0x562d1a0c84f0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa5237fe640 10 client.16863 handle_command_reply: tid=0 2024-03-18T15:45:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 client.16863 shutdown 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 msgr2=0x562d1a0c3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009ca0 tx=0x7fa53c045070 comp rx=0 tx=0).stop 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 shutdown_connections 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a0c84f0 0x562d1a0ca950 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c3bf0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b2d30 0x562d1a0bcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa5237fe640 0 client.16863 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/501222037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 >> 172.21.15.96:0/501222037 conn(0x562d1a0ad8b0 msgr2=0x562d1a0acbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 shutdown_connections 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 1 -- 172.21.15.96:0/501222037 wait complete. 2024-03-18T15:45:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.681+0000 7fa54d3d3740 20 client.16863 trim_cache size 0 max 16384 2024-03-18T15:45:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.698+0000 7fa54d3d3740 1 Processor -- start 2024-03-18T15:45:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.698+0000 7fa54d3d3740 1 -- start start 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.698+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 0x562d1a0c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441a8d20 con 0x562d1a0b2d30 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441aa8f0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441a93f0 con 0x562d1a0b2340 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 0x562d1a0c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 0x562d1a0c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42408/0 (socket says 172.21.15.96:42408) 2024-03-18T15:45:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa5437fe640 1 -- 172.21.15.96:0/2057693001 learned_addr learned my addr 172.21.15.96:0/2057693001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa543fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 -- 172.21.15.96:0/2057693001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 msgr2=0x562d1a0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 -- 172.21.15.96:0/2057693001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 msgr2=0x562d1a0c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 0x562d1a0c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.699+0000 7fa548ca6640 1 -- 172.21.15.96:0/2057693001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1a09f140 con 0x562d1a0b7ea0 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.700+0000 7fa548ca6640 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0b1050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa53c057a40 tx=0x7fa53c032b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5061f25edb580583 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.700+0000 7fa529ffb640 1 -- 172.21.15.96:0/2057693001 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c039030 con 0x562d1a0b7ea0 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.700+0000 7fa529ffb640 1 -- 172.21.15.96:0/2057693001 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c057cc0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.700+0000 7fa529ffb640 1 -- 172.21.15.96:0/2057693001 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c04d8c0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- 172.21.15.96:0/2057693001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 msgr2=0x562d1a0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0b1050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa53c057a40 tx=0x7fa53c032b80 comp rx=0 tx=0).stop 2024-03-18T15:45:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- 172.21.15.96:0/2057693001 shutdown_connections 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0b1050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2d30 0x562d1a0c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/2057693001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2340 0x562d1a0b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- 172.21.15.96:0/2057693001 >> 172.21.15.96:0/2057693001 conn(0x562d1a0ad8b0 msgr2=0x562d1a0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- 172.21.15.96:0/2057693001 shutdown_connections 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- 172.21.15.96:0/2057693001 wait complete. 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 -- start start 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.701+0000 7fa54d3d3740 1 Processor -- start 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 0x562d1a0c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441aa220 con 0x562d1a0b2340 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441a9b50 con 0x562d1a0b7ea0 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441a6d00 con 0x562d1a0b2d30 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa54d3d3740 10 client.0 _ll_register_callbacks cb 0x7fa534005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa543fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 0x562d1a0c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:56916/0 (socket says 172.21.15.96:56916) 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 -- 172.21.15.96:0/3050705224 learned_addr learned my addr 172.21.15.96:0/3050705224 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa5437fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 0x562d1a0c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48926/0 (socket says 172.21.15.96:48926) 2024-03-18T15:45:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa543fff640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0c1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42424/0 (socket says 172.21.15.96:42424) 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 -- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 msgr2=0x562d1a0c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 0x562d1a0c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 -- 172.21.15.96:0/3050705224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 msgr2=0x562d1a0c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 -- 172.21.15.96:0/3050705224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1a0c3240 con 0x562d1a0b7ea0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa548ca6640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0578d0 tx=0x7fa53c02c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0cbb0c296580662 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa5237fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.702+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c034ae0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c057cc0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fa53c04d600 con 0x562d1a0b7ea0 2024-03-18T15:45:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa53c06e030 con 0x562d1a0b7ea0 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 10 client.26675 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d1a12ecb0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53c057cc0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fa53c05d9b0 con 0x562d1a0b7ea0 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 10 client.26675 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 20 client.26675 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a12f530 0x562d1a131950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 4 client.26675 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.703+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d1a01f2d0 con 0x562d1a12f530 2024-03-18T15:45:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.704+0000 7fa543fff640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a12f530 0x562d1a131950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.704+0000 7fa5237fe640 10 client.26675 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:24.704+0000 7fa543fff640 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a12f530 0x562d1a131950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a71bdccf90e03e server_cookie=6357671c7108d03c in_seq=0 out_seq=0 2024-03-18T15:45:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:24 smithi137 ceph-mon[35868]: pgmap v511: 97 pgs: 97 active+clean; 640 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-18T15:45:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:24 smithi193 ceph-mon[36702]: pgmap v511: 97 pgs: 97 active+clean; 640 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-18T15:45:25.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:24 smithi096 ceph-mon[31930]: pgmap v511: 97 pgs: 97 active+clean; 640 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 301 op/s 2024-03-18T15:45:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa5237fe640 1 -- 172.21.15.96:0/3050705224 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d1a01f2d0 con 0x562d1a12f530 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa5237fe640 10 client.26675 handle_command_reply: tid=0 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 client.26675 shutdown 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 msgr2=0x562d1a0c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa53c0578d0 tx=0x7fa53c02c900 comp rx=0 tx=0).stop 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 shutdown_connections 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d1a0b7ea0 0x562d1a0c2c50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d1a0b2d30 0x562d1a0c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562d1a12f530 0x562d1a131950 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 --2- 172.21.15.96:0/3050705224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562d1a0b2340 0x562d1a0c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa5237fe640 0 client.26675 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 >> 172.21.15.96:0/3050705224 conn(0x562d1a0ad8b0 msgr2=0x562d1a0bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 shutdown_connections 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 1 -- 172.21.15.96:0/3050705224 wait complete. 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.455+0000 7fa54d3d3740 20 client.26675 trim_cache size 0 max 16384 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:45:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:45:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 msgr2=0x7fa52c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 0x7fa52c0a4d60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa53c005e30 tx=0x7fa53c0323b0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 msgr2=0x7fa544197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa53400e9c0 tx=0x7fa53400ee80 comp rx=0 tx=0).stop 2024-03-18T15:45:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 shutdown_connections 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa52c0a28a0 0x7fa52c0a4d60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5441033e0 0x7fa54419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa544102a20 0x7fa544197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 --2- 172.21.15.96:0/3449899227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa544101830 0x7fa544197a30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 >> 172.21.15.96:0/3449899227 conn(0x7fa5440fcda0 msgr2=0x7fa544105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 shutdown_connections 2024-03-18T15:45:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:25.456+0000 7fa54a709640 1 -- 172.21.15.96:0/3449899227 wait complete. 2024-03-18T15:45:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:26 smithi137 ceph-mon[35868]: pgmap v512: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-03-18T15:45:27.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:26 smithi193 ceph-mon[36702]: pgmap v512: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-03-18T15:45:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:26 smithi096 ceph-mon[31930]: pgmap v512: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 392 op/s 2024-03-18T15:45:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:28 smithi137 ceph-mon[35868]: pgmap v513: 97 pgs: 97 active+clean; 665 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-03-18T15:45:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:28 smithi193 ceph-mon[36702]: pgmap v513: 97 pgs: 97 active+clean; 665 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-03-18T15:45:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:28 smithi096 ceph-mon[31930]: pgmap v513: 97 pgs: 97 active+clean; 665 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 383 op/s 2024-03-18T15:45:30.069 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:29 smithi193 ceph-mon[36702]: pgmap v514: 97 pgs: 97 active+clean; 684 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-03-18T15:45:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:29 smithi137 ceph-mon[35868]: pgmap v514: 97 pgs: 97 active+clean; 684 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-03-18T15:45:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:29 smithi096 ceph-mon[31930]: pgmap v514: 97 pgs: 97 active+clean; 684 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.6 MiB/s wr, 614 op/s 2024-03-18T15:45:31.331 INFO:tasks.workunit.client.0.smithi096.stdout: 136 1307787 189 954508 221 765890 621 2024-03-18T15:45:31.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:45:31.762 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:45:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:32 smithi193 ceph-mon[36702]: pgmap v515: 97 pgs: 97 active+clean; 683 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-03-18T15:45:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:32 smithi096 ceph-mon[31930]: pgmap v515: 97 pgs: 97 active+clean; 683 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-03-18T15:45:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:32 smithi137 ceph-mon[35868]: pgmap v515: 97 pgs: 97 active+clean; 683 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 593 op/s 2024-03-18T15:45:33.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 -- 172.21.15.96:0/4148103016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4108a50 msgr2=0x7f1ef4108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 --2- 172.21.15.96:0/4148103016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4108a50 0x7f1ef4108e70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400aa10 tx=0x7f1ee402be50 comp rx=0 tx=0).stop 2024-03-18T15:45:33.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 -- 172.21.15.96:0/4148103016 shutdown_connections 2024-03-18T15:45:33.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 --2- 172.21.15.96:0/4148103016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109ee0 0x7f1ef4110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 --2- 172.21.15.96:0/4148103016 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4109440 0x7f1ef41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 --2- 172.21.15.96:0/4148103016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4108a50 0x7f1ef4108e70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.293+0000 7f1ef9150640 1 -- 172.21.15.96:0/4148103016 >> 172.21.15.96:0/4148103016 conn(0x7f1ef406e600 msgr2=0x7f1ef4070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.299+0000 7f1ef9150640 1 -- 172.21.15.96:0/4148103016 shutdown_connections 2024-03-18T15:45:33.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.299+0000 7f1ef9150640 1 -- 172.21.15.96:0/4148103016 wait complete. 2024-03-18T15:45:33.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 Processor -- start 2024-03-18T15:45:33.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 -- start start 2024-03-18T15:45:33.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 0x7f1ef41a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 0x7f1ef41a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 0x7f1ef419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41138a0 con 0x7f1ef4109440 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4113720 con 0x7f1ef4108a50 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.300+0000 7f1ef9150640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef4113a20 con 0x7f1ef4109ee0 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.301+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 0x7f1ef419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.301+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 0x7f1ef419e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48950/0 (socket says 172.21.15.96:48950) 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.301+0000 7f1ef894f640 1 -- 172.21.15.96:0/1317764290 learned_addr learned my addr 172.21.15.96:0/1317764290 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.301+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 0x7f1ef41a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.301+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 0x7f1ef41a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef3fff640 1 -- 172.21.15.96:0/1317764290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 msgr2=0x7f1ef419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 0x7f1ef419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef3fff640 1 -- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 msgr2=0x7f1ef41a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 0x7f1ef41a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef3fff640 1 -- 172.21.15.96:0/1317764290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef419ea90 con 0x7f1ef4108a50 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.302+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 0x7f1ef41a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:33.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.303+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 0x7f1ef41a15d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4036040 tx=0x7f1ee40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.339+0000 7f1ef17fa640 1 -- 172.21.15.96:0/1317764290 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4043020 con 0x7f1ef4108a50 2024-03-18T15:45:33.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.341+0000 7f1ef17fa640 1 -- 172.21.15.96:0/1317764290 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4034bb0 con 0x7f1ef4108a50 2024-03-18T15:45:33.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.341+0000 7f1ef17fa640 1 -- 172.21.15.96:0/1317764290 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee403e7b0 con 0x7f1ef4108a50 2024-03-18T15:45:33.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.341+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef41a2070 con 0x7f1ef4108a50 2024-03-18T15:45:33.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.341+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef41a2540 con 0x7f1ef4108a50 2024-03-18T15:45:33.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.348+0000 7f1efcf9e740 1 Processor -- start 2024-03-18T15:45:33.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.348+0000 7f1efcf9e740 1 -- start start 2024-03-18T15:45:33.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.348+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab42157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.348+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215d70 0x562ab421a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.348+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ab417a500 con 0x562ab4215380 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ab4122be0 con 0x562ab4215d70 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ab41765a0 con 0x562ab421aee0 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab42157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48958/0 (socket says 172.21.15.96:48958) 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.349+0000 7f1ef37fe640 1 -- 172.21.15.96:0/1594835522 learned_addr learned my addr 172.21.15.96:0/1594835522 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef37fe640 1 -- 172.21.15.96:0/1594835522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215d70 msgr2=0x562ab421a870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215d70 0x562ab421a870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef37fe640 1 -- 172.21.15.96:0/1594835522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 msgr2=0x562ab42157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab42157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef37fe640 1 -- 172.21.15.96:0/1594835522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab41364b0 con 0x562ab421aee0 2024-03-18T15:45:33.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab42157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:33.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.350+0000 7f1ef17fa640 1 -- 172.21.15.96:0/1317764290 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f1ee4046020 con 0x7f1ef4108a50 2024-03-18T15:45:33.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.351+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0104a0 tx=0x7f1eec010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539a9ef7ca0c380b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.352+0000 7f1ef17fa640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 0x7f1edc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.352+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 0x7f1edc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.353+0000 7f1ef17fa640 1 -- 172.21.15.96:0/1317764290 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1ee40f08a0 con 0x7f1ef4108a50 2024-03-18T15:45:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.353+0000 7f1eda7fc640 1 -- 172.21.15.96:0/1594835522 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec017070 con 0x562ab421aee0 2024-03-18T15:45:33.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.353+0000 7f1eda7fc640 1 -- 172.21.15.96:0/1594835522 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eec011be0 con 0x562ab421aee0 2024-03-18T15:45:33.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.354+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1594835522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 msgr2=0x562ab421d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.354+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0104a0 tx=0x7f1eec010960 comp rx=0 tx=0).stop 2024-03-18T15:45:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.358+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 0x7f1edc0a4ed0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562ab42165d0 tx=0x7f1ee800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.358+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1594835522 shutdown_connections 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.358+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab421d370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.358+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215d70 0x562ab421a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.358+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1594835522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab42157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.359+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1594835522 >> 172.21.15.96:0/1594835522 conn(0x562ab42108f0 msgr2=0x562ab4210cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.359+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1594835522 shutdown_connections 2024-03-18T15:45:33.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.359+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1594835522 wait complete. 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.359+0000 7f1efcf9e740 1 -- start start 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.359+0000 7f1efcf9e740 1 Processor -- start 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab421f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 0x562ab421fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ab421dee0 con 0x562ab4215d70 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562ab4136160 con 0x562ab4215380 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ab421e060 con 0x562ab421aee0 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1efcf9e740 10 client.0 _ll_register_callbacks cb 0x562ab411a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:33.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48974/0 (socket says 172.21.15.96:48974) 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1ef894f640 1 -- 172.21.15.96:0/1440676763 learned_addr learned my addr 172.21.15.96:0/1440676763 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 0x562ab421fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.360+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab421f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 -- 172.21.15.96:0/1440676763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 msgr2=0x562ab421f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab421f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 -- 172.21.15.96:0/1440676763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 msgr2=0x562ab421fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 0x562ab421fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 -- 172.21.15.96:0/1440676763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab3f587d0 con 0x562ab421aee0 2024-03-18T15:45:33.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 0x562ab421fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:33.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.361+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801bc20 tx=0x7f1ee8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1f3655e5d9659 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.378+0000 7f1ed7ff7640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:45:33.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.378+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8021050 con 0x562ab421aee0 2024-03-18T15:45:33.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.378+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eec011be0 con 0x562ab421aee0 2024-03-18T15:45:33.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.378+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1ee8026640 con 0x562ab421aee0 2024-03-18T15:45:33.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ee800dc90 con 0x562ab421aee0 2024-03-18T15:45:33.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 10 client.26725 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:33.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562ab3e8e3f0 con 0x562ab421aee0 2024-03-18T15:45:33.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1eec011be0 con 0x562ab421aee0 2024-03-18T15:45:33.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1ee80035f0 con 0x562ab421aee0 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 10 client.26725 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 20 client.26725 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.379+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab422b840 0x562ab422dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.380+0000 7f1efcf9e740 4 client.26725 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.380+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ab4187a40 con 0x562ab422b840 2024-03-18T15:45:33.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.380+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab422b840 0x562ab422dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.381+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab422b840 0x562ab422dcc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ac99d97c9ee6e32 server_cookie=b8dcafa15dad33ff in_seq=0 out_seq=0 2024-03-18T15:45:33.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.381+0000 7f1ed7ff7640 10 client.26725 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:33.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.382+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1440676763 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562ab4187a40 con 0x562ab422b840 2024-03-18T15:45:33.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1ed7ff7640 10 client.26725 handle_command_reply: tid=0 2024-03-18T15:45:33.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 client.26725 shutdown 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 msgr2=0x562ab4226ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ee801bc20 tx=0x7f1ee8004730 comp rx=0 tx=0).stop 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 shutdown_connections 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab421aee0 0x562ab4226ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab422b840 0x562ab422dcc0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215d70 0x562ab421fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1440676763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab421f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.394+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 >> 172.21.15.96:0/1440676763 conn(0x562ab42108f0 msgr2=0x562ab420fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.396+0000 7f1ed7ff7640 0 client.26725 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:33.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.396+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 shutdown_connections 2024-03-18T15:45:33.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.396+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1440676763 wait complete. 2024-03-18T15:45:33.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.397+0000 7f1efcf9e740 20 client.26725 trim_cache size 0 max 16384 2024-03-18T15:45:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.478+0000 7f1efcf9e740 1 Processor -- start 2024-03-18T15:45:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.478+0000 7f1efcf9e740 1 -- start start 2024-03-18T15:45:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.478+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.478+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab4217840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.478+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab4217840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab4217840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48976/0 (socket says 172.21.15.96:48976) 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:56952/0 (socket says 172.21.15.96:56952) 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef894f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42456/0 (socket says 172.21.15.96:42456) 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1ef37fe640 1 -- 172.21.15.96:0/278011737 learned_addr learned my addr 172.21.15.96:0/278011737 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41c03e0 con 0x562ab421aee0 2024-03-18T15:45:33.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41c1fb0 con 0x562ab4215380 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.479+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41c0ab0 con 0x562ab4215d70 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 -- 172.21.15.96:0/278011737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 msgr2=0x562ab4217840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab4217840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 -- 172.21.15.96:0/278011737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 msgr2=0x562ab4217300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 -- 172.21.15.96:0/278011737 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ab3e4e0f0 con 0x562ab421aee0 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:33.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.480+0000 7f1ef894f640 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8005bc0 tx=0x7f1ee800de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8612e7426dd035cb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.486+0000 7f1ed9ffb640 1 -- 172.21.15.96:0/278011737 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8021040 con 0x562ab421aee0 2024-03-18T15:45:33.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.486+0000 7f1ed9ffb640 1 -- 172.21.15.96:0/278011737 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee802b070 con 0x562ab421aee0 2024-03-18T15:45:33.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.489+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 msgr2=0x562ab4226e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.489+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8005bc0 tx=0x7f1ee800de30 comp rx=0 tx=0).stop 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 shutdown_connections 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab421aee0 0x562ab4226e90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab4217840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/278011737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab4215380 0x562ab4217300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 >> 172.21.15.96:0/278011737 conn(0x562ab42108f0 msgr2=0x562ab421ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 shutdown_connections 2024-03-18T15:45:33.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 -- 172.21.15.96:0/278011737 wait complete. 2024-03-18T15:45:33.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.494+0000 7f1efcf9e740 1 -- start start 2024-03-18T15:45:33.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.495+0000 7f1efcf9e740 1 Processor -- start 2024-03-18T15:45:33.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.495+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.495+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1ef3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42458/0 (socket says 172.21.15.96:42458) 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab42238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1efcf9e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 0x562ab4223e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1ef3fff640 1 -- 172.21.15.96:0/1285645203 learned_addr learned my addr 172.21.15.96:0/1285645203 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41c18e0 con 0x562ab4215380 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41c1210 con 0x562ab421aee0 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef41a9390 con 0x562ab4215d70 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1ef37fe640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab42238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.496+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 0x562ab4223e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 -- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 msgr2=0x562ab42238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab42238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 -- 172.21.15.96:0/1285645203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 msgr2=0x562ab4226df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1efcf9e740 10 client.0 _ll_register_callbacks cb 0x7f1ee8009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ab41419d0 con 0x562ab421aee0 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee802b070 con 0x562ab421aee0 2024-03-18T15:45:33.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.497+0000 7f1ef894f640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 0x562ab4223e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8003bd0 tx=0x7f1ee80035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3185891fa5c577b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.510+0000 7f1ed7ff7640 10 client.26690 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:45:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.510+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1285645203 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8021070 con 0x562ab421aee0 2024-03-18T15:45:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.510+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1285645203 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f1ee8005230 con 0x562ab421aee0 2024-03-18T15:45:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.510+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1285645203 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ee801be60 con 0x562ab421aee0 2024-03-18T15:45:33.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.510+0000 7f1efcf9e740 10 client.26690 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:33.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({fsmap=34}) -- 0x562ab4290a90 con 0x562ab421aee0 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1285645203 <== mon.2 v2:172.21.15.137:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1ee804e030 con 0x562ab421aee0 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 10 client.26690 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 20 client.26690 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab4291260 0x562ab42936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:33.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 4 client.26690 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:45:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.520+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ab4181d90 con 0x562ab4291260 2024-03-18T15:45:33.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.521+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab4291260 0x562ab42936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.521+0000 7f1ef3fff640 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab4291260 0x562ab42936c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d54a72b2c37b17 server_cookie=24115e2ad035a014 in_seq=0 out_seq=0 2024-03-18T15:45:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.521+0000 7f1ed7ff7640 10 client.26690 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.521+0000 7f1ed7ff7640 1 -- 172.21.15.96:0/1285645203 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562ab4181d90 con 0x562ab4291260 2024-03-18T15:45:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.521+0000 7f1ed7ff7640 10 client.26690 handle_command_reply: tid=0 2024-03-18T15:45:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.522+0000 7f1efcf9e740 1 client.26690 shutdown 2024-03-18T15:45:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 msgr2=0x562ab4223e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 0x562ab4223e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8003bd0 tx=0x7f1ee80035f0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 shutdown_connections 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562ab421aee0 0x562ab4223e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ab4215d70 0x562ab42238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562ab4291260 0x562ab42936c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 --2- 172.21.15.96:0/1285645203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ab4215380 0x562ab4226df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 >> 172.21.15.96:0/1285645203 conn(0x562ab42108f0 msgr2=0x562ab4217300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.526+0000 7f1ed7ff7640 0 client.26690 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:33.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.529+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 shutdown_connections 2024-03-18T15:45:33.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.529+0000 7f1efcf9e740 1 -- 172.21.15.96:0/1285645203 wait complete. 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.529+0000 7f1efcf9e740 20 client.26690 trim_cache size 0 max 16384 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "f8e45139-760e-4c72-8b48-5597834c9d02": { 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "f8e45139-760e-4c72-8b48-5597834c9d02", 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:45:33.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:45:33.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:45:33.531 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 msgr2=0x7f1edc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 0x7f1edc0a4ed0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x562ab42165d0 tx=0x7f1ee800f040 comp rx=0 tx=0).stop 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 msgr2=0x7f1ef41a15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 0x7f1ef41a15d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4036040 tx=0x7f1ee40045e0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 shutdown_connections 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f1edc0a2a10 0x7f1edc0a4ed0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ef4109ee0 0x7f1ef419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ef4109440 0x7f1ef41a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 --2- 172.21.15.96:0/1317764290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ef4108a50 0x7f1ef41a15d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.543+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 >> 172.21.15.96:0/1317764290 conn(0x7f1ef406e600 msgr2=0x7f1ef410cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.544+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 shutdown_connections 2024-03-18T15:45:33.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:45:33.544+0000 7f1ef9150640 1 -- 172.21.15.96:0/1317764290 wait complete. 2024-03-18T15:45:33.597 INFO:tasks.cephfs.filesystem:scrub status for tag:f8e45139-760e-4c72-8b48-5597834c9d02 - {'path': '/', 'tag': 'f8e45139-760e-4c72-8b48-5597834c9d02', 'options': 'recursive,force'} 2024-03-18T15:45:33.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:33 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:45:33.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:33 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:45:34.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:33 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:45:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:34 smithi193 ceph-mon[36702]: pgmap v516: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-03-18T15:45:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:34 smithi096 ceph-mon[31930]: pgmap v516: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-03-18T15:45:35.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:34 smithi137 ceph-mon[35868]: pgmap v516: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 632 op/s 2024-03-18T15:45:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:45:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:45:35.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:45:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:45:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:45:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:45:36.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:45:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:45:36.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:45:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:36 smithi193 ceph-mon[36702]: pgmap v517: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-03-18T15:45:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:36 smithi096 ceph-mon[31930]: pgmap v517: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-03-18T15:45:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:36 smithi137 ceph-mon[35868]: pgmap v517: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 558 op/s 2024-03-18T15:45:38.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:38 smithi193 ceph-mon[36702]: pgmap v518: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2024-03-18T15:45:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:38 smithi096 ceph-mon[31930]: pgmap v518: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2024-03-18T15:45:39.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:38 smithi137 ceph-mon[35868]: pgmap v518: 97 pgs: 97 active+clean; 687 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2024-03-18T15:45:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.172+0000 7f54629bc640 1 -- 172.21.15.96:0/3881909635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 msgr2=0x7f545c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.172+0000 7f54629bc640 1 --2- 172.21.15.96:0/3881909635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c10e1a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5454009ca0 tx=0x7f545402c120 comp rx=0 tx=0).stop 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.190+0000 7f54629bc640 1 -- 172.21.15.96:0/3881909635 shutdown_connections 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.191+0000 7f54629bc640 1 --2- 172.21.15.96:0/3881909635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c10e1a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.191+0000 7f54629bc640 1 --2- 172.21.15.96:0/3881909635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c073410 0x7f545c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.191+0000 7f54629bc640 1 --2- 172.21.15.96:0/3881909635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c072ab0 0x7f545c072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.191+0000 7f54629bc640 1 -- 172.21.15.96:0/3881909635 >> 172.21.15.96:0/3881909635 conn(0x7f545c06e650 msgr2=0x7f545c070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.191+0000 7f54629bc640 1 -- 172.21.15.96:0/3881909635 shutdown_connections 2024-03-18T15:45:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.197+0000 7f54629bc640 1 -- 172.21.15.96:0/3881909635 wait complete. 2024-03-18T15:45:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 Processor -- start 2024-03-18T15:45:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 -- start start 2024-03-18T15:45:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c072ab0 0x7f545c114de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c1b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f545c11c1b0 con 0x7f545c072ab0 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f545c11c030 con 0x7f545c073410 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.198+0000 7f54629bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f545c11c330 con 0x7f545c109510 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:35640/0 (socket says 172.21.15.96:35640) 2024-03-18T15:45:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 -- 172.21.15.96:0/2998947350 learned_addr learned my addr 172.21.15.96:0/2998947350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f5460f59640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c1b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 -- 172.21.15.96:0/2998947350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 msgr2=0x7f545c1b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c1b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 -- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c072ab0 msgr2=0x7f545c114de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:45:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c072ab0 0x7f545c114de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.199+0000 7f545b7fe640 1 -- 172.21.15.96:0/2998947350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c1184a0 con 0x7f545c073410 2024-03-18T15:45:40.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.200+0000 7f545b7fe640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f545000b470 tx=0x7f545000b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.210+0000 7f54597fa640 1 -- 172.21.15.96:0/2998947350 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5450015020 con 0x7f545c073410 2024-03-18T15:45:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.210+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c1b6ec0 con 0x7f545c073410 2024-03-18T15:45:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.210+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c1b72d0 con 0x7f545c073410 2024-03-18T15:45:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.214+0000 7f546559b740 1 Processor -- start 2024-03-18T15:45:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 -- start start 2024-03-18T15:45:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcbb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aabfc18a40 con 0x55aabfcb3ef0 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aabfbc1920 con 0x55aabfcb9060 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f546559b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aabfc14ae0 con 0x55aabfcb3500 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f54597fa640 1 -- 172.21.15.96:0/2998947350 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5450011d70 con 0x7f545c073410 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f5460f59640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f5460f59640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:55034/0 (socket says 172.21.15.96:55034) 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f5460f59640 1 -- 172.21.15.96:0/2334375919 learned_addr learned my addr 172.21.15.96:0/2334375919 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.215+0000 7f54597fa640 1 -- 172.21.15.96:0/2998947350 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545000f340 con 0x7f545c073410 2024-03-18T15:45:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.216+0000 7f5460f59640 1 -- 172.21.15.96:0/2334375919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 msgr2=0x55aabfcbb4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:45:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.216+0000 7f5460f59640 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcbb4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.216+0000 7f5460f59640 1 -- 172.21.15.96:0/2334375919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 msgr2=0x55aabfcb89f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:45:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.217+0000 7f54597fa640 1 -- 172.21.15.96:0/2998947350 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5450018020 con 0x7f545c073410 2024-03-18T15:45:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.217+0000 7f545bfff640 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.217+0000 7f5460f59640 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcb89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.218+0000 7f54597fa640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 0x7f54440a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.218+0000 7f54597fa640 1 -- 172.21.15.96:0/2998947350 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f54500cf610 con 0x7f545c073410 2024-03-18T15:45:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.220+0000 7f5460f59640 1 -- 172.21.15.96:0/2334375919 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabfbd9460 con 0x55aabfcb3500 2024-03-18T15:45:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.220+0000 7f545bfff640 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:45:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.221+0000 7f5460f59640 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5454032660 tx=0x7f5454032a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c24c948bda29aa9d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.221+0000 7f545bfff640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 0x7f54440a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.222+0000 7f545bfff640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 0x7f54440a4ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55aabfcb42f0 tx=0x7f544c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.244+0000 7f544a7fc640 1 -- 172.21.15.96:0/2334375919 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454043020 con 0x55aabfcb3500 2024-03-18T15:45:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.244+0000 7f544a7fc640 1 -- 172.21.15.96:0/2334375919 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5454003e50 con 0x55aabfcb3500 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 -- 172.21.15.96:0/2334375919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 msgr2=0x55aabfcb3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5454032660 tx=0x7f5454032a00 comp rx=0 tx=0).stop 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 -- 172.21.15.96:0/2334375919 shutdown_connections 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcbb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 --2- 172.21.15.96:0/2334375919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcb3920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 -- 172.21.15.96:0/2334375919 >> 172.21.15.96:0/2334375919 conn(0x55aabfcaea70 msgr2=0x55aabfcaee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 -- 172.21.15.96:0/2334375919 shutdown_connections 2024-03-18T15:45:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.248+0000 7f546559b740 1 -- 172.21.15.96:0/2334375919 wait complete. 2024-03-18T15:45:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 -- start start 2024-03-18T15:45:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 Processor -- start 2024-03-18T15:45:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcbdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcbe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aabfcbc000 con 0x55aabfcb3ef0 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55aabfbd4d70 con 0x55aabfcb9060 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aabfcbc180 con 0x55aabfcb3500 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.249+0000 7f546559b740 10 client.0 _ll_register_callbacks cb 0x55aabf92ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.250+0000 7f5460f59640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.250+0000 7f5460f59640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:35642/0 (socket says 172.21.15.96:35642) 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.250+0000 7f5460f59640 1 -- 172.21.15.96:0/3912387244 learned_addr learned my addr 172.21.15.96:0/3912387244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.250+0000 7f545b7fe640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcbe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 -- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 msgr2=0x55aabfcbdb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcbdb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 -- 172.21.15.96:0/3912387244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 msgr2=0x55aabfcbe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcbe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 -- 172.21.15.96:0/3912387244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabf92d210 con 0x55aabfcb9060 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f545b7fe640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcbe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.251+0000 7f5460f59640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5454009ca0 tx=0x7f5454009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b645460fa2e2f722 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f543f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:45:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545404c070 con 0x55aabfcb9060 2024-03-18T15:45:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5454003b70 con 0x55aabfcb9060 2024-03-18T15:45:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aabfc25f80 con 0x55aabfcb9060 2024-03-18T15:45:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5454058050 con 0x55aabfcb9060 2024-03-18T15:45:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f546559b740 10 client.26699 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aabf9f9620 con 0x55aabfcb9060 2024-03-18T15:45:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.275+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55aabfc25f80 con 0x55aabfcb9060 2024-03-18T15:45:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f545403f970 con 0x55aabfcb9060 2024-03-18T15:45:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 10 client.26699 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 10 client.26699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 20 client.26699 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfcc9a30 0x55aabfccbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 4 client.26699 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aabfc202d0 con 0x55aabfcc9a30 2024-03-18T15:45:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.276+0000 7f545bfff640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfcc9a30 0x55aabfccbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.277+0000 7f545bfff640 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfcc9a30 0x55aabfccbe50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0ac64a9c9831c6 server_cookie=7a5755060dc26b78 in_seq=0 out_seq=0 2024-03-18T15:45:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.277+0000 7f543f7fe640 10 client.26699 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.279+0000 7f543f7fe640 1 -- 172.21.15.96:0/3912387244 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55aabfc202d0 con 0x55aabfcc9a30 2024-03-18T15:45:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.279+0000 7f543f7fe640 10 client.26699 handle_command_reply: tid=0 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.279+0000 7f546559b740 1 client.26699 shutdown 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.279+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 msgr2=0x55aabfcc5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.279+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5454009ca0 tx=0x7f5454009890 comp rx=0 tx=0).stop 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 shutdown_connections 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc5030 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfcc9a30 0x55aabfccbe50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3ef0 0x55aabfcbe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 --2- 172.21.15.96:0/3912387244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3500 0x55aabfcbdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 >> 172.21.15.96:0/3912387244 conn(0x55aabfcaea70 msgr2=0x55aabfcadc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.280+0000 7f543f7fe640 0 client.26699 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.281+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 shutdown_connections 2024-03-18T15:45:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.281+0000 7f546559b740 1 -- 172.21.15.96:0/3912387244 wait complete. 2024-03-18T15:45:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.281+0000 7f546559b740 20 client.26699 trim_cache size 0 max 16384 2024-03-18T15:45:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.426+0000 7f546559b740 1 Processor -- start 2024-03-18T15:45:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.426+0000 7f546559b740 1 -- start start 2024-03-18T15:45:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.426+0000 7f546559b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f546559b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f546559b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 0x55aabfcbce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f545bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f545bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:35656/0 (socket says 172.21.15.96:35656) 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f545bfff640 1 -- 172.21.15.96:0/3891349505 learned_addr learned my addr 172.21.15.96:0/3891349505 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f544000e620 con 0x55aabfcb9060 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f54400101f0 con 0x55aabfcb3500 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f544000ecf0 con 0x55aabfcb3ef0 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f545b7fe640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.427+0000 7f5460f59640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 0x55aabfcbce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f545bfff640 1 -- 172.21.15.96:0/3891349505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 msgr2=0x55aabfcc4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f545bfff640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f545bfff640 1 -- 172.21.15.96:0/3891349505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 msgr2=0x55aabfcbce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f545bfff640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 0x55aabfcbce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f545bfff640 1 -- 172.21.15.96:0/3891349505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabfcb3140 con 0x55aabfcb3500 2024-03-18T15:45:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.431+0000 7f5460f59640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 0x55aabfcbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.432+0000 7f545bfff640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f544c010b60 tx=0x7f544c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b6788b1a715686 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.432+0000 7f545b7fe640 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.438+0000 7f5449ffb640 1 -- 172.21.15.96:0/3891349505 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c02c070 con 0x55aabfcb3500 2024-03-18T15:45:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.439+0000 7f5449ffb640 1 -- 172.21.15.96:0/3891349505 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544c012e30 con 0x55aabfcb3500 2024-03-18T15:45:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.439+0000 7f5449ffb640 1 -- 172.21.15.96:0/3891349505 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c01f680 con 0x55aabfcb3500 2024-03-18T15:45:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.440+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 msgr2=0x55aabfcb23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.440+0000 7f546559b740 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f544c010b60 tx=0x7f544c016000 comp rx=0 tx=0).stop 2024-03-18T15:45:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.445+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 shutdown_connections 2024-03-18T15:45:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.445+0000 7f546559b740 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb9060 0x55aabfcbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.445+0000 7f546559b740 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.445+0000 7f546559b740 1 --2- 172.21.15.96:0/3891349505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb3500 0x55aabfcb23e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.445+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 >> 172.21.15.96:0/3891349505 conn(0x55aabfcaea70 msgr2=0x55aabfcb1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.448 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.448+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 shutdown_connections 2024-03-18T15:45:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.448+0000 7f546559b740 1 -- 172.21.15.96:0/3891349505 wait complete. 2024-03-18T15:45:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.449+0000 7f546559b740 1 -- start start 2024-03-18T15:45:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.449+0000 7f546559b740 1 Processor -- start 2024-03-18T15:45:40.450 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.450+0000 7f546559b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f545bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f545bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44470/0 (socket says 172.21.15.96:44470) 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f545bfff640 1 -- 172.21.15.96:0/450479387 learned_addr learned my addr 172.21.15.96:0/450479387 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f545b7fe640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f544000fb20 con 0x55aabfcb3500 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f544000f450 con 0x55aabfcb9060 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f544000c600 con 0x55aabfcb3ef0 2024-03-18T15:45:40.452 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f5460f59640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.451+0000 7f546559b740 10 client.0 _ll_register_callbacks cb 0x7f544c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 -- 172.21.15.96:0/450479387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 msgr2=0x55aabfcc2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 -- 172.21.15.96:0/450479387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 msgr2=0x55aabfcc4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabfd2dd80 con 0x55aabfcb3ef0 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545bfff640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:40.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.452+0000 7f545b7fe640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc1a40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f545009bc20 tx=0x7f545009b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f493fc876d0422ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.471+0000 7f543f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:45:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.471+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500b8020 con 0x55aabfcb3ef0 2024-03-18T15:45:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.471+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5450011a90 con 0x55aabfcb3ef0 2024-03-18T15:45:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.471+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544c012e30 con 0x55aabfcb3ef0 2024-03-18T15:45:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f545009bf20 con 0x55aabfcb3ef0 2024-03-18T15:45:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f546559b740 10 client.26746 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:40.473 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aabfd2f150 con 0x55aabfcb3ef0 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54500a0070 con 0x55aabfcb3ef0 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f545009af00 con 0x55aabfcb3ef0 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f546559b740 10 client.26746 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f546559b740 10 client.26746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.472+0000 7f546559b740 20 client.26746 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.473+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfd2fbd0 0x55aabfd31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.473+0000 7f546559b740 4 client.26746 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:45:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.473+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f544c012e30 con 0x55aabfd2fbd0 2024-03-18T15:45:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.473+0000 7f545bfff640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfd2fbd0 0x55aabfd31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.474+0000 7f545bfff640 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfd2fbd0 0x55aabfd31ff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b00fe2ff0822a0 server_cookie=b6ad580682298df3 in_seq=0 out_seq=0 2024-03-18T15:45:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.475+0000 7f543f7fe640 10 client.26746 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.861+0000 7f543f7fe640 1 -- 172.21.15.96:0/450479387 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f544c012e30 con 0x55aabfd2fbd0 2024-03-18T15:45:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.861+0000 7f543f7fe640 10 client.26746 handle_command_reply: tid=0 2024-03-18T15:45:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.861+0000 7f546559b740 1 client.26746 shutdown 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 msgr2=0x55aabfcc1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc1a40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f545009bc20 tx=0x7f545009b880 comp rx=0 tx=0).stop 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 shutdown_connections 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55aabfcb9060 0x55aabfcc2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabfcb3ef0 0x55aabfcc1a40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55aabfd2fbd0 0x55aabfd31ff0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 --2- 172.21.15.96:0/450479387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aabfcb3500 0x55aabfcc4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 >> 172.21.15.96:0/450479387 conn(0x55aabfcaea70 msgr2=0x55aabfcb2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f543f7fe640 0 client.26746 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 shutdown_connections 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.864+0000 7f546559b740 1 -- 172.21.15.96:0/450479387 wait complete. 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.865+0000 7f546559b740 20 client.26746 trim_cache size 0 max 16384 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:45:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 msgr2=0x7f54440a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 0x7f54440a4ec0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x55aabfcb42f0 tx=0x7f544c005f90 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 msgr2=0x7f545c115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f545000b470 tx=0x7f545000b930 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 shutdown_connections 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f54440a2a00 0x7f54440a4ec0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f545c109510 0x7f545c1b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f545c073410 0x7f545c115320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.896+0000 7f54629bc640 1 --2- 172.21.15.96:0/2998947350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f545c072ab0 0x7f545c114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.897+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 >> 172.21.15.96:0/2998947350 conn(0x7f545c06e650 msgr2=0x7f545c0701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:40.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.897+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 shutdown_connections 2024-03-18T15:45:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:40.897+0000 7f54629bc640 1 -- 172.21.15.96:0/2998947350 wait complete. 2024-03-18T15:45:40.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:40 smithi193 ceph-mon[36702]: pgmap v519: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 446 op/s 2024-03-18T15:45:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:40 smithi096 ceph-mon[31930]: pgmap v519: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 446 op/s 2024-03-18T15:45:41.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:40 smithi137 ceph-mon[35868]: pgmap v519: 97 pgs: 97 active+clean; 685 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 446 op/s 2024-03-18T15:45:41.331 INFO:tasks.workunit.client.0.smithi096.stdout: 142 2156794 438 1579149 288 1292418 1216 2024-03-18T15:45:42.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:42 smithi193 ceph-mon[36702]: pgmap v520: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-03-18T15:45:42.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:42 smithi096 ceph-mon[31930]: pgmap v520: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-03-18T15:45:43.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:42 smithi137 ceph-mon[35868]: pgmap v520: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 235 op/s 2024-03-18T15:45:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:44 smithi193 ceph-mon[36702]: pgmap v521: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 250 op/s 2024-03-18T15:45:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:44 smithi096 ceph-mon[31930]: pgmap v521: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 250 op/s 2024-03-18T15:45:45.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:44 smithi137 ceph-mon[35868]: pgmap v521: 97 pgs: 97 active+clean; 690 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 250 op/s 2024-03-18T15:45:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:46 smithi193 ceph-mon[36702]: pgmap v522: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-18T15:45:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:46 smithi096 ceph-mon[31930]: pgmap v522: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-18T15:45:47.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:46 smithi137 ceph-mon[35868]: pgmap v522: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-03-18T15:45:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:48 smithi193 ceph-mon[36702]: pgmap v523: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-03-18T15:45:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:48 smithi096 ceph-mon[31930]: pgmap v523: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-03-18T15:45:49.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:48 smithi137 ceph-mon[35868]: pgmap v523: 97 pgs: 97 active+clean; 686 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-03-18T15:45:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:50 smithi193 ceph-mon[36702]: pgmap v524: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2024-03-18T15:45:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:50 smithi096 ceph-mon[31930]: pgmap v524: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2024-03-18T15:45:51.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:50 smithi137 ceph-mon[35868]: pgmap v524: 97 pgs: 97 active+clean; 678 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 417 op/s 2024-03-18T15:45:51.331 INFO:tasks.workunit.client.0.smithi096.stdout: 145 890512 225 651644 201 541143 686 2024-03-18T15:45:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:52 smithi096 ceph-mon[31930]: pgmap v525: 97 pgs: 97 active+clean; 688 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 453 op/s 2024-03-18T15:45:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:52 smithi137 ceph-mon[35868]: pgmap v525: 97 pgs: 97 active+clean; 688 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 453 op/s 2024-03-18T15:45:53.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:52 smithi193 ceph-mon[36702]: pgmap v525: 97 pgs: 97 active+clean; 688 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 453 op/s 2024-03-18T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.145+0000 7fe29ff37640 1 -- 172.21.15.96:0/1794377761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980ffac0 msgr2=0x7fe2980ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.145+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1794377761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980ffac0 0x7fe2980ffee0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe294009930 tx=0x7fe29402bc30 comp rx=0 tx=0).stop 2024-03-18T15:45:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.146+0000 7fe29ff37640 1 -- 172.21.15.96:0/1794377761 shutdown_connections 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.146+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1794377761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe298103450 0x7fe298109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.146+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1794377761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.146+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1794377761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2980ffac0 0x7fe2980ffee0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.146+0000 7fe29ff37640 1 -- 172.21.15.96:0/1794377761 >> 172.21.15.96:0/1794377761 conn(0x7fe2980fb430 msgr2=0x7fe2980fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.147+0000 7fe29ff37640 1 -- 172.21.15.96:0/1794377761 shutdown_connections 2024-03-18T15:45:54.148 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.147+0000 7fe29ff37640 1 -- 172.21.15.96:0/1794377761 wait complete. 2024-03-18T15:45:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.147+0000 7fe29ff37640 1 Processor -- start 2024-03-18T15:45:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 -- start start 2024-03-18T15:45:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2980ffac0 0x7fe298195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe29810b2f0 con 0x7fe298103450 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe29810b170 con 0x7fe298102a90 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29ff37640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe29810b470 con 0x7fe2980ffac0 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29e4d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29e4d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43190/0 (socket says 172.21.15.96:43190) 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29e4d4640 1 -- 172.21.15.96:0/1468780267 learned_addr learned my addr 172.21.15.96:0/1468780267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2980ffac0 0x7fe298195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.148+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 -- 172.21.15.96:0/1468780267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2980ffac0 msgr2=0x7fe298195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2980ffac0 0x7fe298195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 -- 172.21.15.96:0/1468780267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 msgr2=0x7fe298195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 -- 172.21.15.96:0/1468780267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29819d4e0 con 0x7fe298103450 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe29e4d4640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe29000eb10 tx=0x7fe29000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe28effd640 1 -- 172.21.15.96:0/1468780267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290004b20 con 0x7fe298103450 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe28effd640 1 -- 172.21.15.96:0/1468780267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe290004cc0 con 0x7fe298103450 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.149+0000 7fe28effd640 1 -- 172.21.15.96:0/1468780267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290010760 con 0x7fe298103450 2024-03-18T15:45:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.150+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29819d7d0 con 0x7fe298103450 2024-03-18T15:45:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.150+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29819dcd0 con 0x7fe298103450 2024-03-18T15:45:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.153+0000 7fe28effd640 1 -- 172.21.15.96:0/1468780267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe290005020 con 0x7fe298103450 2024-03-18T15:45:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.153+0000 7fe2a2bf7740 1 Processor -- start 2024-03-18T15:45:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.153+0000 7fe2a2bf7740 1 -- start start 2024-03-18T15:45:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.153+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dbf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.153+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dc7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555781d24a40 con 0x555781dbf340 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555781ccd920 con 0x555781dbfd30 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555781d20ae0 con 0x555781dc4ea0 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:37850/0 (socket says 172.21.15.96:37850) 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 -- 172.21.15.96:0/1540644198 learned_addr learned my addr 172.21.15.96:0/1540644198 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dbf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe28effd640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 0x7fe2840a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 -- 172.21.15.96:0/1540644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 msgr2=0x555781dc7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dc7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 -- 172.21.15.96:0/1540644198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 msgr2=0x555781dbf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dbf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe29dcd3640 1 -- 172.21.15.96:0/1540644198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555781ce5460 con 0x555781dbfd30 2024-03-18T15:45:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.154+0000 7fe28effd640 1 -- 172.21.15.96:0/1468780267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe2900cf030 con 0x7fe298103450 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.155+0000 7fe29e4d4640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 0x7fe2840a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.155+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe294000c00 tx=0x7fe294005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b3f50748690720 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.155+0000 7fe283fff640 1 -- 172.21.15.96:0/1540644198 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294045020 con 0x555781dbfd30 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.155+0000 7fe29e4d4640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 0x7fe2840a9330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe290098370 tx=0x7fe2900058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.156+0000 7fe283fff640 1 -- 172.21.15.96:0/1540644198 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294004040 con 0x555781dbfd30 2024-03-18T15:45:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.156+0000 7fe283fff640 1 -- 172.21.15.96:0/1540644198 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294005610 con 0x555781dbfd30 2024-03-18T15:45:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1540644198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 msgr2=0x555781dc4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe294000c00 tx=0x7fe294005b50 comp rx=0 tx=0).stop 2024-03-18T15:45:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1540644198 shutdown_connections 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dc7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc4830 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1540644198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dbf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.157+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1540644198 >> 172.21.15.96:0/1540644198 conn(0x555781dba8b0 msgr2=0x555781dbacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.158+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1540644198 shutdown_connections 2024-03-18T15:45:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.158+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1540644198 wait complete. 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.158+0000 7fe2a2bf7740 1 -- start start 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.158+0000 7fe2a2bf7740 1 Processor -- start 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dc9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dd0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555781dc7df0 con 0x555781dbf340 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe29d4d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dc9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe29d4d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dc9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43206/0 (socket says 172.21.15.96:43206) 2024-03-18T15:45:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe29d4d2640 1 -- 172.21.15.96:0/1599458107 learned_addr learned my addr 172.21.15.96:0/1599458107 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555781ce0d70 con 0x555781dbfd30 2024-03-18T15:45:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555781dc7f70 con 0x555781dc4ea0 2024-03-18T15:45:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.159+0000 7fe2a2bf7740 10 client.0 _ll_register_callbacks cb 0x555781a38a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dd0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 -- 172.21.15.96:0/1599458107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 msgr2=0x555781dc9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 -- 172.21.15.96:0/1599458107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 msgr2=0x555781dc9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dc9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.160+0000 7fe29d4d2640 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555781a39210 con 0x555781dc4ea0 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.161+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dd0bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe288010e40 tx=0x7fe2880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13634a916a10a87 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.161+0000 7fe2817fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.161+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28801a040 con 0x555781dc4ea0 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.161+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe288025c80 con 0x555781dc4ea0 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.161+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe294004040 con 0x555781dc4ea0 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.162+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe288004030 con 0x555781dc4ea0 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.162+0000 7fe2a2bf7740 10 client.26755 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.162+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555781b05620 con 0x555781dc4ea0 2024-03-18T15:45:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.163+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2880041d0 con 0x555781dc4ea0 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fe288026050 con 0x555781dc4ea0 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 10 client.26755 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 10 client.26755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 20 client.26755 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781dd5440 0x555781dd7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 4 client.26755 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe294004040 con 0x555781dd5440 2024-03-18T15:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.164+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781dd5440 0x555781dd7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.165+0000 7fe2817fa640 10 client.26755 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.165+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781dd5440 0x555781dd7890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7429e49b041a6d8 server_cookie=79a0536e330b6e72 in_seq=0 out_seq=0 2024-03-18T15:45:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.166+0000 7fe2817fa640 1 -- 172.21.15.96:0/1599458107 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7fe294004040 con 0x555781dd5440 2024-03-18T15:45:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.166+0000 7fe2817fa640 10 client.26755 handle_command_reply: tid=0 2024-03-18T15:45:54.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.166+0000 7fe2a2bf7740 1 client.26755 shutdown 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 msgr2=0x555781dd0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dd0bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe288010e40 tx=0x7fe2880046c0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 shutdown_connections 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781dd0bb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781dc9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781dd5440 0x555781dd7890 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/1599458107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbf340 0x555781dc9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 >> 172.21.15.96:0/1599458107 conn(0x555781dba8b0 msgr2=0x555781db9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2817fa640 0 client.26755 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.167+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 shutdown_connections 2024-03-18T15:45:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.168+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/1599458107 wait complete. 2024-03-18T15:45:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.168+0000 7fe2a2bf7740 20 client.26755 trim_cache size 0 max 16384 2024-03-18T15:45:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 Processor -- start 2024-03-18T15:45:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 -- start start 2024-03-18T15:45:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbf340 0x555781dbe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781ddc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a6b00 con 0x555781dbfd30 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a86d0 con 0x555781dbf340 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.197+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a71d0 con 0x555781dc4ea0 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43222/0 (socket says 172.21.15.96:43222) 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 -- 172.21.15.96:0/3792418091 learned_addr learned my addr 172.21.15.96:0/3792418091 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbf340 0x555781dbe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781ddc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 -- 172.21.15.96:0/3792418091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 msgr2=0x555781ddc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781ddc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 -- 172.21.15.96:0/3792418091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbf340 msgr2=0x555781dbe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbf340 0x555781dbe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 -- 172.21.15.96:0/3792418091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555781dbb620 con 0x555781dbfd30 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781ddc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe294005c40 tx=0x7fe294005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3629a3f6f6f18541 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe2837fe640 1 -- 172.21.15.96:0/3792418091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294045040 con 0x555781dbfd30 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe2837fe640 1 -- 172.21.15.96:0/3792418091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294009520 con 0x555781dbfd30 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.198+0000 7fe2837fe640 1 -- 172.21.15.96:0/3792418091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294003a40 con 0x555781dbfd30 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/3792418091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 msgr2=0x555781ddbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe294005c40 tx=0x7fe294005d50 comp rx=0 tx=0).stop 2024-03-18T15:45:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/3792418091 shutdown_connections 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dc4ea0 0x555781ddc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dbfd30 0x555781ddbe90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/3792418091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbf340 0x555781dbe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/3792418091 >> 172.21.15.96:0/3792418091 conn(0x555781dba8b0 msgr2=0x555781dbdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/3792418091 shutdown_connections 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.199+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/3792418091 wait complete. 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 -- start start 2024-03-18T15:45:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 Processor -- start 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dbf340 0x555781e338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781e33e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a8000 con 0x555781dc4ea0 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a7930 con 0x555781dbfd30 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2981a4ae0 con 0x555781dbf340 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.200+0000 7fe2a2bf7740 10 client.0 _ll_register_callbacks cb 0x7fe288010700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.205 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43232/0 (socket says 172.21.15.96:43232) 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 -- 172.21.15.96:0/508243458 learned_addr learned my addr 172.21.15.96:0/508243458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781e33e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dbf340 0x555781e338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 -- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dbf340 msgr2=0x555781e338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dbf340 0x555781e338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 -- 172.21.15.96:0/508243458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 msgr2=0x555781e33e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781e33e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.201+0000 7fe29d4d2640 1 -- 172.21.15.96:0/508243458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555781e3add0 con 0x555781dc4ea0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe29d4d2640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe2880264f0 tx=0x7fe288009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd453b27234c25d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2817fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288037030 con 0x555781dc4ea0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe294009520 con 0x555781dc4ea0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fe288025a80 con 0x555781dc4ea0 2024-03-18T15:45:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe288047030 con 0x555781dc4ea0 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2a2bf7740 10 client.16914 fetch_fsmap learned FSMap version 34 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.202+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555781e3c0f0 con 0x555781dc4ea0 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe294009520 con 0x555781dc4ea0 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fe28803d070 con 0x555781dc4ea0 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 10 client.16914 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 20 client.16914 populate_metadata read hostname 'smithi096' 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781e3cb00 0x555781e3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 4 client.16914 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555781d31f80 con 0x555781e3cb00 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.203+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781e3cb00 0x555781e3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.204+0000 7fe2817fa640 10 client.16914 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.204+0000 7fe29dcd3640 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781e3cb00 0x555781e3ef20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d54c20fd9f3d77 server_cookie=1c7d90747d37be0d in_seq=0 out_seq=0 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.763+0000 7fe2817fa640 1 -- 172.21.15.96:0/508243458 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555781d31f80 con 0x555781e3cb00 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2817fa640 10 client.16914 handle_command_reply: tid=0 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 client.16914 shutdown 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 msgr2=0x555781e3a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe2880264f0 tx=0x7fe288009e90 comp rx=0 tx=0).stop 2024-03-18T15:45:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 shutdown_connections 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x555781e3cb00 0x555781e3ef20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555781dc4ea0 0x555781e3a790 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555781dbfd30 0x555781e33e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2817fa640 0 client.16914 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 --2- 172.21.15.96:0/508243458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555781dbf340 0x555781e338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 >> 172.21.15.96:0/508243458 conn(0x555781dba8b0 msgr2=0x555781dbe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 shutdown_connections 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 1 -- 172.21.15.96:0/508243458 wait complete. 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.764+0000 7fe2a2bf7740 20 client.16914 trim_cache size 0 max 16384 2024-03-18T15:45:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:45:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:45:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:45:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:45:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 msgr2=0x7fe2840a9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 0x7fe2840a9330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe290098370 tx=0x7fe2900058d0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 msgr2=0x7fe29819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe29000eb10 tx=0x7fe29000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 shutdown_connections 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2840a6e70 0x7fe2840a9330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe298103450 0x7fe29819ce70 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe298102a90 0x7fe298195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 --2- 172.21.15.96:0/1468780267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2980ffac0 0x7fe298195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 >> 172.21.15.96:0/1468780267 conn(0x7fe2980fb430 msgr2=0x7fe298105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 shutdown_connections 2024-03-18T15:45:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:45:54.765+0000 7fe29ff37640 1 -- 172.21.15.96:0/1468780267 wait complete. 2024-03-18T15:45:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:54 smithi096 ceph-mon[31930]: pgmap v526: 97 pgs: 97 active+clean; 716 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 629 op/s 2024-03-18T15:45:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:54 smithi137 ceph-mon[35868]: pgmap v526: 97 pgs: 97 active+clean; 716 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 629 op/s 2024-03-18T15:45:55.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:54 smithi193 ceph-mon[36702]: pgmap v526: 97 pgs: 97 active+clean; 716 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 629 op/s 2024-03-18T15:45:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:56 smithi096 ceph-mon[31930]: pgmap v527: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 716 op/s 2024-03-18T15:45:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:56 smithi137 ceph-mon[35868]: pgmap v527: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 716 op/s 2024-03-18T15:45:57.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:56 smithi193 ceph-mon[36702]: pgmap v527: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 716 op/s 2024-03-18T15:45:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:45:58 smithi096 ceph-mon[31930]: pgmap v528: 97 pgs: 97 active+clean; 720 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 688 op/s 2024-03-18T15:45:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:45:58 smithi137 ceph-mon[35868]: pgmap v528: 97 pgs: 97 active+clean; 720 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 688 op/s 2024-03-18T15:45:59.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:45:58 smithi193 ceph-mon[36702]: pgmap v528: 97 pgs: 97 active+clean; 720 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 688 op/s 2024-03-18T15:46:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:00 smithi096 ceph-mon[31930]: pgmap v529: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 784 op/s 2024-03-18T15:46:00.968 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:00 smithi193 ceph-mon[36702]: pgmap v529: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 784 op/s 2024-03-18T15:46:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:00 smithi137 ceph-mon[35868]: pgmap v529: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.9 MiB/s wr, 784 op/s 2024-03-18T15:46:01.331 INFO:tasks.workunit.client.0.smithi096.stdout: 149 1491009 391 1082900 243 888609 755 2024-03-18T15:46:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:02 smithi096 ceph-mon[31930]: pgmap v530: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 600 op/s 2024-03-18T15:46:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:02 smithi137 ceph-mon[35868]: pgmap v530: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 600 op/s 2024-03-18T15:46:03.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:02 smithi193 ceph-mon[36702]: pgmap v530: 97 pgs: 97 active+clean; 721 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 600 op/s 2024-03-18T15:46:03.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:46:03.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:46:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:04 smithi137 ceph-mon[35868]: pgmap v531: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-03-18T15:46:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:04 smithi193 ceph-mon[36702]: pgmap v531: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-03-18T15:46:05.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:04 smithi096 ceph-mon[31930]: pgmap v531: 97 pgs: 97 active+clean; 722 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 562 op/s 2024-03-18T15:46:05.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.727+0000 7f5847a4b640 1 -- 172.21.15.96:0/1778219147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840072b50 msgr2=0x7f5840072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.727+0000 7f5847a4b640 1 --2- 172.21.15.96:0/1778219147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840072b50 0x7f5840072f70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5838009ca0 tx=0x7f583802c120 comp rx=0 tx=0).stop 2024-03-18T15:46:05.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.735+0000 7f5847a4b640 1 -- 172.21.15.96:0/1778219147 shutdown_connections 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.735+0000 7f5847a4b640 1 --2- 172.21.15.96:0/1778219147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f584010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.735+0000 7f5847a4b640 1 --2- 172.21.15.96:0/1778219147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840108e20 0x7f58401092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.735+0000 7f5847a4b640 1 --2- 172.21.15.96:0/1778219147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840072b50 0x7f5840072f70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.735+0000 7f5847a4b640 1 -- 172.21.15.96:0/1778219147 >> 172.21.15.96:0/1778219147 conn(0x7f584006e600 msgr2=0x7f5840070a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.736+0000 7f5847a4b640 1 -- 172.21.15.96:0/1778219147 shutdown_connections 2024-03-18T15:46:05.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.736+0000 7f5847a4b640 1 -- 172.21.15.96:0/1778219147 wait complete. 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.737+0000 7f5847a4b640 1 Processor -- start 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.739+0000 7f5847a4b640 1 -- start start 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.739+0000 7f5847a4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 0x7f58401a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.739+0000 7f5847a4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.739+0000 7f5847a4b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f58401add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5847a4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f584011c1b0 con 0x7f5840108e20 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5847a4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f584011c030 con 0x7f5840072b50 2024-03-18T15:46:05.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5847a4b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f584011c330 con 0x7f58401097e0 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f58457e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 0x7f58401a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5844fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5844fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49758/0 (socket says 172.21.15.96:49758) 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5844fe6640 1 -- 172.21.15.96:0/2080224934 learned_addr learned my addr 172.21.15.96:0/2080224934 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.740+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f58401add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 -- 172.21.15.96:0/2080224934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 msgr2=0x7f58401add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f58401add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 -- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 msgr2=0x7f58401a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 -- 172.21.15.96:0/2080224934 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58401ae390 con 0x7f5840072b50 2024-03-18T15:46:05.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f58401add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:05.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f5844fe6640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:05.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.741+0000 7f58457e7640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 0x7f58401a6770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5838009bf0 tx=0x7f5838035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:05.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.750+0000 7f58367fc640 1 -- 172.21.15.96:0/2080224934 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838039040 con 0x7f5840072b50 2024-03-18T15:46:05.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.751+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58401ae620 con 0x7f5840072b50 2024-03-18T15:46:05.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.751+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58401aeae0 con 0x7f5840072b50 2024-03-18T15:46:05.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.755+0000 7f58367fc640 1 -- 172.21.15.96:0/2080224934 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5838044070 con 0x7f5840072b50 2024-03-18T15:46:05.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.755+0000 7f58367fc640 1 -- 172.21.15.96:0/2080224934 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583803f440 con 0x7f5840072b50 2024-03-18T15:46:05.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 Processor -- start 2024-03-18T15:46:05.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 -- start start 2024-03-18T15:46:05.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f58367fc640 1 -- 172.21.15.96:0/2080224934 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f583803f6a0 con 0x7f5840072b50 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 0x5564cef32780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef37850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564cee97500 con 0x5564cef32360 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5564cee3fbe0 con 0x5564cef37ec0 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.760+0000 7f584a63c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564cee935a0 con 0x5564cef32d50 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.761+0000 7f5844fe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.761+0000 7f5844fe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33344/0 (socket says 172.21.15.96:33344) 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.761+0000 7f5844fe6640 1 -- 172.21.15.96:0/2975692078 learned_addr learned my addr 172.21.15.96:0/2975692078 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.761+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 0x5564cef32780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 -- 172.21.15.96:0/2975692078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 msgr2=0x5564cef37850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef37850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 -- 172.21.15.96:0/2975692078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 msgr2=0x5564cef3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 -- 172.21.15.96:0/2975692078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564cee534b0 con 0x5564cef32360 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.762+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 0x5564cef32780 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fa0 tx=0x7f583c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91449734175cedd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:05.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.764+0000 7f58367fc640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 0x7f582c0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.764+0000 7f58457e7640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 0x7f582c0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.765+0000 7f58457e7640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 0x7f582c0a4ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5838032490 tx=0x7f583803d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:05.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.765+0000 7f58367fc640 1 -- 172.21.15.96:0/2080224934 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f58380f1160 con 0x7f5840072b50 2024-03-18T15:46:05.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.774+0000 7f582b7fe640 1 -- 172.21.15.96:0/2975692078 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c004310 con 0x5564cef32360 2024-03-18T15:46:05.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.775+0000 7f582b7fe640 1 -- 172.21.15.96:0/2975692078 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c0044b0 con 0x5564cef32360 2024-03-18T15:46:05.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 -- 172.21.15.96:0/2975692078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 msgr2=0x5564cef32780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 0x5564cef32780 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f583c009fa0 tx=0x7f583c0118d0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 -- 172.21.15.96:0/2975692078 shutdown_connections 2024-03-18T15:46:05.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef37850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 --2- 172.21.15.96:0/2975692078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32360 0x5564cef32780 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.776+0000 7f584a63c740 1 -- 172.21.15.96:0/2975692078 >> 172.21.15.96:0/2975692078 conn(0x5564cef2d8d0 msgr2=0x5564cef2dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.785+0000 7f584a63c740 1 -- 172.21.15.96:0/2975692078 shutdown_connections 2024-03-18T15:46:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.785+0000 7f584a63c740 1 -- 172.21.15.96:0/2975692078 wait complete. 2024-03-18T15:46:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 -- start start 2024-03-18T15:46:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 Processor -- start 2024-03-18T15:46:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 0x5564cef3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef41dd0 0x5564cef441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564cef3aec0 con 0x5564cef37ec0 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.786+0000 7f584a63c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5564cee53160 con 0x5564cef41dd0 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f584a63c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564cef3b040 con 0x5564cef32d50 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f584a63c740 10 client.0 _ll_register_callbacks cb 0x5564cee37150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f5844fe6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 0x5564cef3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef3c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:44994/0 (socket says 172.21.15.96:44994) 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f5845fe8640 1 -- 172.21.15.96:0/2326974583 learned_addr learned my addr 172.21.15.96:0/2326974583 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.787+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef41dd0 0x5564cef441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 -- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 msgr2=0x5564cef3c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef3c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 -- 172.21.15.96:0/2326974583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef41dd0 msgr2=0x5564cef441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef41dd0 0x5564cef441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 -- 172.21.15.96:0/2326974583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cec757d0 con 0x5564cef37ec0 2024-03-18T15:46:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.788+0000 7f5844fe6640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 0x5564cef3cda0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5564cef2e840 tx=0x7f583000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e80bd7b322e5fbc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:05.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.796+0000 7f5828ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:05.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.796+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830011dd0 con 0x5564cef37ec0 2024-03-18T15:46:05.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.796+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564ceea4a40 con 0x5564cef37ec0 2024-03-18T15:46:05.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.796+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f583000fc90 con 0x5564cef37ec0 2024-03-18T15:46:05.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.798+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5830023030 con 0x5564cef37ec0 2024-03-18T15:46:05.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 10 client.16926 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:05.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5564cebab3f0 con 0x5564cef37ec0 2024-03-18T15:46:05.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5564ceea4a40 con 0x5564cef37ec0 2024-03-18T15:46:05.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f5830028510 con 0x5564cef37ec0 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 10 client.16926 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 20 client.16926 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cef48d20 0x5564cef4b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 4 client.16926 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.799+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564cee9ed90 con 0x5564cef48d20 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.800+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cef48d20 0x5564cef4b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.801+0000 7f5845fe8640 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cef48d20 0x5564cef4b160 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7bbe3cee1e62434 server_cookie=bca81d7c8fb24bd7 in_seq=0 out_seq=0 2024-03-18T15:46:05.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f5828ff9640 10 client.16926 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:05.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f5828ff9640 1 -- 172.21.15.96:0/2326974583 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5564cee9ed90 con 0x5564cef48d20 2024-03-18T15:46:05.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f5828ff9640 10 client.16926 handle_command_reply: tid=0 2024-03-18T15:46:05.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 client.16926 shutdown 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 msgr2=0x5564cef3cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 0x5564cef3cda0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5564cef2e840 tx=0x7f583000f610 comp rx=0 tx=0).stop 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 shutdown_connections 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef41dd0 0x5564cef441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cef48d20 0x5564cef4b160 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef37ec0 0x5564cef3cda0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 --2- 172.21.15.96:0/2326974583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef32d50 0x5564cef3c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f5828ff9640 0 client.16926 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:05.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.803+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 >> 172.21.15.96:0/2326974583 conn(0x5564cef2d8d0 msgr2=0x5564cef32570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:05.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.805+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 shutdown_connections 2024-03-18T15:46:05.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.805+0000 7f584a63c740 1 -- 172.21.15.96:0/2326974583 wait complete. 2024-03-18T15:46:05.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.806+0000 7f584a63c740 20 client.16926 trim_cache size 0 max 16384 2024-03-18T15:46:05.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.953+0000 7f584a63c740 1 Processor -- start 2024-03-18T15:46:05.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.953+0000 7f584a63c740 1 -- start start 2024-03-18T15:46:05.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.955+0000 7f584a63c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.955+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.955+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49780/0 (socket says 172.21.15.96:49780) 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.955+0000 7f5845fe8640 1 -- 172.21.15.96:0/1084365010 learned_addr learned my addr 172.21.15.96:0/1084365010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef2dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f582400e670 con 0x5564cef32d50 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5824010240 con 0x5564cef37ec0 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f582400ed40 con 0x5564cef41dd0 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f5845fe8640 1 -- 172.21.15.96:0/1084365010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 msgr2=0x5564cef2dcb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:46:05.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f5845fe8640 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef2dcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f5845fe8640 1 -- 172.21.15.96:0/1084365010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 msgr2=0x5564cef3ad50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:46:05.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.956+0000 7f5845fe8640 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3ad50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.957+0000 7f5845fe8640 1 -- 172.21.15.96:0/1084365010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ceb6b0f0 con 0x5564cef32d50 2024-03-18T15:46:05.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.957+0000 7f5845fe8640 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f583c01b040 tx=0x7f583c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8051a5f0e154f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:05.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.963+0000 7f582affd640 1 -- 172.21.15.96:0/1084365010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c034070 con 0x5564cef32d50 2024-03-18T15:46:05.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.964+0000 7f582affd640 1 -- 172.21.15.96:0/1084365010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c01e040 con 0x5564cef32d50 2024-03-18T15:46:05.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.964+0000 7f582affd640 1 -- 172.21.15.96:0/1084365010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c01d040 con 0x5564cef32d50 2024-03-18T15:46:05.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.965+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 msgr2=0x5564cef3a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:05.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.965+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f583c01b040 tx=0x7f583c005850 comp rx=0 tx=0).stop 2024-03-18T15:46:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.968+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 shutdown_connections 2024-03-18T15:46:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.968+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef2dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.968+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef3ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.969+0000 7f584a63c740 1 --2- 172.21.15.96:0/1084365010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef3a2b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.969+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 >> 172.21.15.96:0/1084365010 conn(0x5564cef2d8d0 msgr2=0x5564cef32570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:05.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.971+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 shutdown_connections 2024-03-18T15:46:05.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.973+0000 7f584a63c740 1 -- 172.21.15.96:0/1084365010 wait complete. 2024-03-18T15:46:05.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.974+0000 7f584a63c740 1 -- start start 2024-03-18T15:46:05.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.974+0000 7f584a63c740 1 Processor -- start 2024-03-18T15:46:05.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.975+0000 7f584a63c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.975+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.975+0000 7f5845fe8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49796/0 (socket says 172.21.15.96:49796) 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.975+0000 7f5845fe8640 1 -- 172.21.15.96:0/3057996530 learned_addr learned my addr 172.21.15.96:0/3057996530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.975+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef5abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f582400fb70 con 0x5564cef32d50 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f5844fe6640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f582400f4a0 con 0x5564cef37ec0 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f582400c650 con 0x5564cef41dd0 2024-03-18T15:46:05.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f5845fe8640 1 -- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 msgr2=0x5564cef5abc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:46:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f5845fe8640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef5abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.976+0000 7f5845fe8640 1 -- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 msgr2=0x5564cef59c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:05.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.977+0000 7f584a63c740 10 client.0 _ll_register_callbacks cb 0x7f58300099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.977+0000 7f5845fe8640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef59c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.977+0000 7f5845fe8640 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cee5e9d0 con 0x5564cef32d50 2024-03-18T15:46:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.977+0000 7f5844fe6640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.978+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c01e040 con 0x5564cef32d50 2024-03-18T15:46:05.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:05.978+0000 7f5845fe8640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f583c003b70 tx=0x7f583c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8537130330a85c0e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:06.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.007+0000 7f5828ff9640 10 client.16932 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.007+0000 7f5828ff9640 1 -- 172.21.15.96:0/3057996530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c034040 con 0x5564cef32d50 2024-03-18T15:46:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.008+0000 7f5828ff9640 1 -- 172.21.15.96:0/3057996530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f583c004fd0 con 0x5564cef32d50 2024-03-18T15:46:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.008+0000 7f5828ff9640 1 -- 172.21.15.96:0/3057996530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f583c01d040 con 0x5564cef32d50 2024-03-18T15:46:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.008+0000 7f584a63c740 10 client.16932 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:06.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.008+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5564cef5c1f0 con 0x5564cef32d50 2024-03-18T15:46:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f5828ff9640 1 -- 172.21.15.96:0/3057996530 <== mon.0 v2:172.21.15.96:3300/0 4 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f583c040030 con 0x5564cef32d50 2024-03-18T15:46:06.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 10 client.16932 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 20 client.16932 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cefaea60 0x5564cefb0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 4 client.16932 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.012+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f583c01e040 con 0x5564cefaea60 2024-03-18T15:46:06.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.013+0000 7f5844fe6640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cefaea60 0x5564cefb0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.013+0000 7f5844fe6640 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cefaea60 0x5564cefb0e50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7593220bc6f5bc51 server_cookie=24177b957307e209 in_seq=0 out_seq=0 2024-03-18T15:46:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.013+0000 7f5828ff9640 10 client.16932 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.013+0000 7f5828ff9640 1 -- 172.21.15.96:0/3057996530 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f583c01e040 con 0x5564cefaea60 2024-03-18T15:46:06.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.014+0000 7f5828ff9640 10 client.16932 handle_command_reply: tid=0 2024-03-18T15:46:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.014+0000 7f584a63c740 1 client.16932 shutdown 2024-03-18T15:46:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.015+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 msgr2=0x5564cef596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:06.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.015+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f583c003b70 tx=0x7f583c0118d0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 shutdown_connections 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564cef41dd0 0x5564cef5abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564cef37ec0 0x5564cef59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5564cefaea60 0x5564cefb0e50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 --2- 172.21.15.96:0/3057996530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564cef32d50 0x5564cef596c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 >> 172.21.15.96:0/3057996530 conn(0x5564cef2d8d0 msgr2=0x5564cef2dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f5828ff9640 0 client.16932 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 shutdown_connections 2024-03-18T15:46:06.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 1 -- 172.21.15.96:0/3057996530 wait complete. 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.016+0000 7f584a63c740 20 client.16932 trim_cache size 0 max 16384 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "f8e45139-760e-4c72-8b48-5597834c9d02": { 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "f8e45139-760e-4c72-8b48-5597834c9d02", 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:46:06.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:46:06.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:46:06.019 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.034+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 msgr2=0x7f582c0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.034+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 0x7f582c0a4ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5838032490 tx=0x7f583803d000 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 msgr2=0x7f58401a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 0x7f58401a6770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5838009bf0 tx=0x7f5838035d10 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 shutdown_connections 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f582c0a2a10 0x7f582c0a4ed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58401097e0 0x7f58401add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5840108e20 0x7f58401a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 --2- 172.21.15.96:0/2080224934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5840072b50 0x7f58401a6770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:06.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.035+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 >> 172.21.15.96:0/2080224934 conn(0x7f584006e600 msgr2=0x7f584010a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:06.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.037+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 shutdown_connections 2024-03-18T15:46:06.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:06.037+0000 7f5847a4b640 1 -- 172.21.15.96:0/2080224934 wait complete. 2024-03-18T15:46:06.266 INFO:tasks.cephfs.filesystem:scrub status for tag:f8e45139-760e-4c72-8b48-5597834c9d02 - {'path': '/', 'tag': 'f8e45139-760e-4c72-8b48-5597834c9d02', 'options': 'recursive,force'} 2024-03-18T15:46:07.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:06 smithi137 ceph-mon[35868]: pgmap v532: 97 pgs: 97 active+clean; 725 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2024-03-18T15:46:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:06 smithi193 ceph-mon[36702]: pgmap v532: 97 pgs: 97 active+clean; 725 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2024-03-18T15:46:07.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:06 smithi096 ceph-mon[31930]: pgmap v532: 97 pgs: 97 active+clean; 725 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2849533245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18073410 msgr2=0x7fcf18108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2849533245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18073410 0x7fcf18108e70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009900 tx=0x7fcf0c02bc40 comp rx=0 tx=0).stop 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2849533245 shutdown_connections 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2849533245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18109510 0x7fcf1810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2849533245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18073410 0x7fcf18108e70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2849533245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18072ab0 0x7fcf18072ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2849533245 >> 172.21.15.96:0/2849533245 conn(0x7fcf1806e650 msgr2=0x7fcf18070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2849533245 shutdown_connections 2024-03-18T15:46:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.002+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2849533245 wait complete. 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 Processor -- start 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 -- start start 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18073410 0x7fcf181a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 0x7fcf181adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1811c110 con 0x7fcf18109510 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1811bf90 con 0x7fcf18073410 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.003+0000 7fcf1f09a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1811c290 con 0x7fcf18072ab0 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:45006/0 (socket says 172.21.15.96:45006) 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 -- 172.21.15.96:0/2394893188 learned_addr learned my addr 172.21.15.96:0/2394893188 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 0x7fcf181adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 -- 172.21.15.96:0/2394893188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18073410 msgr2=0x7fcf181a6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf17fff640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18073410 0x7fcf181a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18073410 0x7fcf181a6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 -- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 msgr2=0x7fcf181adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 0x7fcf181adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1ce36640 1 -- 172.21.15.96:0/2394893188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf181ae2b0 con 0x7fcf18072ab0 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.004+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 0x7fcf181adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:46:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.005+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cce0 tx=0x7fcf1000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.023+0000 7fcf15ffb640 1 -- 172.21.15.96:0/2394893188 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000f650 con 0x7fcf18072ab0 2024-03-18T15:46:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.023+0000 7fcf15ffb640 1 -- 172.21.15.96:0/2394893188 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1000fcd0 con 0x7fcf18072ab0 2024-03-18T15:46:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.023+0000 7fcf15ffb640 1 -- 172.21.15.96:0/2394893188 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10018cd0 con 0x7fcf18072ab0 2024-03-18T15:46:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.023+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf181ae5a0 con 0x7fcf18072ab0 2024-03-18T15:46:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.023+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181aeaa0 con 0x7fcf18072ab0 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.028+0000 7fcf15ffb640 1 -- 172.21.15.96:0/2394893188 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fcf10004b20 con 0x7fcf18072ab0 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.028+0000 7fcf21d64740 1 Processor -- start 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.028+0000 7fcf21d64740 1 -- start start 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca337ef0 0x5590ca33c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca33f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590ca29ca40 con 0x5590ca33d060 2024-03-18T15:46:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590ca245920 con 0x5590ca337500 2024-03-18T15:46:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf21d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5590ca298ae0 con 0x5590ca337ef0 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33394/0 (socket says 172.21.15.96:33394) 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 -- 172.21.15.96:0/905776012 learned_addr learned my addr 172.21.15.96:0/905776012 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf17fff640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca33f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca337ef0 0x5590ca33c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 -- 172.21.15.96:0/905776012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca337ef0 msgr2=0x5590ca33c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca337ef0 0x5590ca33c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 -- 172.21.15.96:0/905776012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 msgr2=0x5590ca33f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca33f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.029+0000 7fcf1d637640 1 -- 172.21.15.96:0/905776012 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ca25d460 con 0x5590ca337500 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.030+0000 7fcf17fff640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca33f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:46:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.030+0000 7fcf1d637640 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009bb0 tx=0x7fcf08010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eae82ffab7d4c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.031+0000 7fcf15ffb640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 0x7fcf000a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.031+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 0x7fcf000a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.032+0000 7fcf1ce36640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 0x7fcf000a4ec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5590ca338960 tx=0x7fcf10007f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.032+0000 7fcf15ffb640 1 -- 172.21.15.96:0/2394893188 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcf100cfff0 con 0x7fcf18072ab0 2024-03-18T15:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.068+0000 7fcefeffd640 1 -- 172.21.15.96:0/905776012 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08004020 con 0x5590ca337500 2024-03-18T15:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.068+0000 7fcefeffd640 1 -- 172.21.15.96:0/905776012 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf080041c0 con 0x5590ca337500 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 -- 172.21.15.96:0/905776012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 msgr2=0x5590ca337920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009bb0 tx=0x7fcf08010740 comp rx=0 tx=0).stop 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 -- 172.21.15.96:0/905776012 shutdown_connections 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca33f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca337ef0 0x5590ca33c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 --2- 172.21.15.96:0/905776012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca337920 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.069+0000 7fcf21d64740 1 -- 172.21.15.96:0/905776012 >> 172.21.15.96:0/905776012 conn(0x5590ca332a70 msgr2=0x5590ca332e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.071+0000 7fcf21d64740 1 -- 172.21.15.96:0/905776012 shutdown_connections 2024-03-18T15:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.071+0000 7fcf21d64740 1 -- 172.21.15.96:0/905776012 wait complete. 2024-03-18T15:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.071+0000 7fcf21d64740 1 -- start start 2024-03-18T15:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.071+0000 7fcf21d64740 1 Processor -- start 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca341840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 0x5590ca341d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca348dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5590ca340060 con 0x5590ca337ef0 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5590ca258d70 con 0x5590ca337500 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5590ca3401e0 con 0x5590ca33d060 2024-03-18T15:46:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf21d64740 10 client.0 _ll_register_callbacks cb 0x5590c9fb0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca341840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca341840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33406/0 (socket says 172.21.15.96:33406) 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf1d637640 1 -- 172.21.15.96:0/4021338449 learned_addr learned my addr 172.21.15.96:0/4021338449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf17fff640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 0x5590ca341d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.072+0000 7fcf1d637640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca348dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.073+0000 7fcf17fff640 1 -- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 msgr2=0x5590ca348dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.073+0000 7fcf17fff640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca348dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.073+0000 7fcf17fff640 1 -- 172.21.15.96:0/4021338449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 msgr2=0x5590ca341840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.073+0000 7fcf17fff640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca341840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.073+0000 7fcf17fff640 1 -- 172.21.15.96:0/4021338449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c9fb1210 con 0x5590ca337ef0 2024-03-18T15:46:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.074+0000 7fcf17fff640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 0x5590ca341d80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009850 tx=0x7fcf0c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4a316902be78e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.083+0000 7fcef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.083+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c005d50 con 0x5590ca337ef0 2024-03-18T15:46:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.083+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf080041c0 con 0x5590ca337ef0 2024-03-18T15:46:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.084+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcf0c005230 con 0x5590ca337ef0 2024-03-18T15:46:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.084+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c04d050 con 0x5590ca337ef0 2024-03-18T15:46:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.089+0000 7fcf21d64740 10 client.16938 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.089+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590ca07d620 con 0x5590ca337ef0 2024-03-18T15:46:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.090+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf080041c0 con 0x5590ca337ef0 2024-03-18T15:46:08.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fcf0c003700 con 0x5590ca337ef0 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 10 client.16938 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 10 client.16938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 20 client.16938 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca34da80 0x5590ca34feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 4 client.16938 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590ca2a9f80 con 0x5590ca34da80 2024-03-18T15:46:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.092+0000 7fcf1d637640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca34da80 0x5590ca34feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.093+0000 7fcf1d637640 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca34da80 0x5590ca34feb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9bcd94933f1a4f server_cookie=2cbc7929607ec5e in_seq=0 out_seq=0 2024-03-18T15:46:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.095+0000 7fcef7fff640 10 client.16938 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.095+0000 7fcef7fff640 1 -- 172.21.15.96:0/4021338449 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5590ca2a9f80 con 0x5590ca34da80 2024-03-18T15:46:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.095+0000 7fcef7fff640 10 client.16938 handle_command_reply: tid=0 2024-03-18T15:46:08.097 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.096+0000 7fcf21d64740 1 client.16938 shutdown 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 msgr2=0x5590ca341d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 0x5590ca341d80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009850 tx=0x7fcf0c005a20 comp rx=0 tx=0).stop 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 shutdown_connections 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca348dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca34da80 0x5590ca34feb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337ef0 0x5590ca341d80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 --2- 172.21.15.96:0/4021338449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca341840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 >> 172.21.15.96:0/4021338449 conn(0x5590ca332a70 msgr2=0x5590ca331c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcef7fff640 0 client.16938 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.102+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 shutdown_connections 2024-03-18T15:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.103+0000 7fcf21d64740 1 -- 172.21.15.96:0/4021338449 wait complete. 2024-03-18T15:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.103+0000 7fcf21d64740 20 client.16938 trim_cache size 0 max 16384 2024-03-18T15:46:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.177+0000 7fcf21d64740 1 Processor -- start 2024-03-18T15:46:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 -- start start 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 0x5590ca339170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca348d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b77e0 con 0x5590ca33d060 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b93b0 con 0x5590ca337500 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf21d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b7eb0 con 0x5590ca332a70 2024-03-18T15:46:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 0x5590ca339170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33414/0 (socket says 172.21.15.96:33414) 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf17fff640 1 -- 172.21.15.96:0/311608956 learned_addr learned my addr 172.21.15.96:0/311608956 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf1d637640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 0x5590ca339170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:45030/0 (socket says 172.21.15.96:45030) 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.181+0000 7fcf1d637640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca348d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 -- 172.21.15.96:0/311608956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 msgr2=0x5590ca339170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 0x5590ca339170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 -- 172.21.15.96:0/311608956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 msgr2=0x5590ca348d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca348d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 -- 172.21.15.96:0/311608956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590ca3426e0 con 0x5590ca337500 2024-03-18T15:46:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf1d637640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca348d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:46:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.182+0000 7fcf17fff640 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004650 tx=0x7fcf0c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=418679dfc13de2ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.226+0000 7fcefe7fc640 1 -- 172.21.15.96:0/311608956 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c005d50 con 0x5590ca337500 2024-03-18T15:46:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.227+0000 7fcefe7fc640 1 -- 172.21.15.96:0/311608956 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c005520 con 0x5590ca337500 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.227+0000 7fcefe7fc640 1 -- 172.21.15.96:0/311608956 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c02c2c0 con 0x5590ca337500 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 -- 172.21.15.96:0/311608956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 msgr2=0x5590ca3396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c004650 tx=0x7fcf0c000bc0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 -- 172.21.15.96:0/311608956 shutdown_connections 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca33d060 0x5590ca348d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca337500 0x5590ca3396b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 --2- 172.21.15.96:0/311608956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca332a70 0x5590ca339170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.228+0000 7fcf21d64740 1 -- 172.21.15.96:0/311608956 >> 172.21.15.96:0/311608956 conn(0x5590ca33cb30 msgr2=0x5590ca332e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.232+0000 7fcf21d64740 1 -- 172.21.15.96:0/311608956 shutdown_connections 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.232+0000 7fcf21d64740 1 -- 172.21.15.96:0/311608956 wait complete. 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.232+0000 7fcf21d64740 1 -- start start 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 Processor -- start 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 0x5590ca3468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca347850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b8ce0 con 0x5590ca337500 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b8610 con 0x5590ca332a70 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b58a0 con 0x5590ca33d060 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf21d64740 10 client.0 _ll_register_callbacks cb 0x7fcf08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.233+0000 7fcf17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 0x5590ca3468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf1d637640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33426/0 (socket says 172.21.15.96:33426) 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf1d637640 1 -- 172.21.15.96:0/2332225856 learned_addr learned my addr 172.21.15.96:0/2332225856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca347850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf17fff640 1 -- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 msgr2=0x5590ca347850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf17fff640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca347850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf17fff640 1 -- 172.21.15.96:0/2332225856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 msgr2=0x5590ca346370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf17fff640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf17fff640 1 -- 172.21.15.96:0/2332225856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ca347e40 con 0x5590ca337500 2024-03-18T15:46:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.234+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.235+0000 7fcf17fff640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 0x5590ca3468b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c003aa0 tx=0x7fcf0c003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b2c21fb30722e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcef7fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c056020 con 0x5590ca337500 2024-03-18T15:46:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c005520 con 0x5590ca337500 2024-03-18T15:46:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7fcf0c034850 con 0x5590ca337500 2024-03-18T15:46:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c02c2c0 con 0x5590ca337500 2024-03-18T15:46:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcf21d64740 10 client.16944 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.248+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5590ca3b3df0 con 0x5590ca337500 2024-03-18T15:46:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c005520 con 0x5590ca337500 2024-03-18T15:46:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fcf0c051830 con 0x5590ca337500 2024-03-18T15:46:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 10 client.16944 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 20 client.16944 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca3b4630 0x5590ca3b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 4 client.16944 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590ca2a42d0 con 0x5590ca3b4630 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.252+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca3b4630 0x5590ca3b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.253+0000 7fcef7fff640 10 client.16944 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.253+0000 7fcf1d637640 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca3b4630 0x5590ca3b6a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0552ddf2f55268 server_cookie=5504c86485eeb85b in_seq=0 out_seq=0 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.365+0000 7fcef7fff640 1 -- 172.21.15.96:0/2332225856 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590ca2a42d0 con 0x5590ca3b4630 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.365+0000 7fcef7fff640 10 client.16944 handle_command_reply: tid=0 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.365+0000 7fcf21d64740 1 client.16944 shutdown 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 msgr2=0x5590ca3468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 0x5590ca3468b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c003aa0 tx=0x7fcf0c003ec0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 shutdown_connections 2024-03-18T15:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590ca33d060 0x5590ca347850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5590ca3b4630 0x5590ca3b6a50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5590ca337500 0x5590ca3468b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 --2- 172.21.15.96:0/2332225856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590ca332a70 0x5590ca346370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 >> 172.21.15.96:0/2332225856 conn(0x5590ca33cb30 msgr2=0x5590ca332e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcef7fff640 0 client.16944 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 shutdown_connections 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 1 -- 172.21.15.96:0/2332225856 wait complete. 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.366+0000 7fcf21d64740 20 client.16944 trim_cache size 0 max 16384 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:46:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 msgr2=0x7fcf000a4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 0x7fcf000a4ec0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5590ca338960 tx=0x7fcf10007f90 comp rx=0 tx=0).stop 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 msgr2=0x7fcf181a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cce0 tx=0x7fcf1000a620 comp rx=0 tx=0).stop 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 shutdown_connections 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcf000a2a00 0x7fcf000a4ec0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf18109510 0x7fcf181adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf18073410 0x7fcf181a6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 --2- 172.21.15.96:0/2394893188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf18072ab0 0x7fcf181a6690 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 >> 172.21.15.96:0/2394893188 conn(0x7fcf1806e650 msgr2=0x7fcf180701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 shutdown_connections 2024-03-18T15:46:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:08.383+0000 7fcf1f09a640 1 -- 172.21.15.96:0/2394893188 wait complete. 2024-03-18T15:46:09.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:08 smithi137 ceph-mon[35868]: pgmap v533: 97 pgs: 97 active+clean; 724 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 289 op/s 2024-03-18T15:46:09.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:08 smithi193 ceph-mon[36702]: pgmap v533: 97 pgs: 97 active+clean; 724 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 289 op/s 2024-03-18T15:46:09.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:08 smithi096 ceph-mon[31930]: pgmap v533: 97 pgs: 97 active+clean; 724 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 289 op/s 2024-03-18T15:46:10.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:09 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:10.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:09 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:10.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:09 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:11.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:10 smithi137 ceph-mon[35868]: pgmap v534: 97 pgs: 97 active+clean; 727 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 344 op/s 2024-03-18T15:46:11.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:10 smithi193 ceph-mon[36702]: pgmap v534: 97 pgs: 97 active+clean; 727 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 344 op/s 2024-03-18T15:46:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:10 smithi096 ceph-mon[31930]: pgmap v534: 97 pgs: 97 active+clean; 727 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 344 op/s 2024-03-18T15:46:11.331 INFO:tasks.workunit.client.0.smithi096.stdout: 153 1885012 421 1377502 301 1123060 1147 2024-03-18T15:46:12.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:11 smithi137 ceph-mon[35868]: pgmap v535: 97 pgs: 97 active+clean; 719 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2024-03-18T15:46:12.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:11 smithi193 ceph-mon[36702]: pgmap v535: 97 pgs: 97 active+clean; 719 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2024-03-18T15:46:12.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:11 smithi096 ceph-mon[31930]: pgmap v535: 97 pgs: 97 active+clean; 719 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 268 op/s 2024-03-18T15:46:13.705 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-18T15:46:13.706 INFO:tasks.workunit.client.0.smithi096.stdout:Final score for writes: 153 2024-03-18T15:46:13.706 INFO:tasks.workunit.client.0.smithi096.stdout:Final score for reads : 720353 2024-03-18T15:46:13.706 INFO:tasks.workunit.client.0.smithi096.stdout: 2024-03-18T15:46:13.706 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-18T15:46:13.706 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-18T15:46:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:14 smithi193 ceph-mon[36702]: pgmap v536: 97 pgs: 97 active+clean; 705 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 283 op/s 2024-03-18T15:46:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:14 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:46:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:14 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:46:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:14 smithi096 ceph-mon[31930]: pgmap v536: 97 pgs: 97 active+clean; 705 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 283 op/s 2024-03-18T15:46:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:14 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:46:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:14 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:46:15.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:14 smithi137 ceph-mon[35868]: pgmap v536: 97 pgs: 97 active+clean; 705 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 283 op/s 2024-03-18T15:46:15.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:14 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:46:15.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:14 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:46:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:15 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:46:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:15 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:46:15.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:15 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:46:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:15 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:46:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:15 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:46:15.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:15 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:46:16.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:15 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:46:16.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:15 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:46:16.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:15 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:46:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:16 smithi193 ceph-mon[36702]: pgmap v537: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-18T15:46:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:16 smithi096 ceph-mon[31930]: pgmap v537: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-18T15:46:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:16 smithi137 ceph-mon[35868]: pgmap v537: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 332 op/s 2024-03-18T15:46:18.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:18 smithi193 ceph-mon[36702]: pgmap v538: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-18T15:46:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:18 smithi096 ceph-mon[31930]: pgmap v538: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-18T15:46:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:18 smithi137 ceph-mon[35868]: pgmap v538: 97 pgs: 97 active+clean; 714 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 260 op/s 2024-03-18T15:46:20.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:20 smithi193 ceph-mon[36702]: pgmap v539: 97 pgs: 97 active+clean; 754 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-03-18T15:46:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:20 smithi096 ceph-mon[31930]: pgmap v539: 97 pgs: 97 active+clean; 754 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-03-18T15:46:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:20 smithi137 ceph-mon[35868]: pgmap v539: 97 pgs: 97 active+clean; 754 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 607 op/s 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- 172.21.15.96:0/2454644257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 msgr2=0x7f4470108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2454644257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f4470108b40 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4468009ca0 tx=0x7f446802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- 172.21.15.96:0/2454644257 shutdown_connections 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2454644257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f4470108b40 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2454644257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44701018f0 0x7f4470101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2454644257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4470100700 0x7f4470100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- 172.21.15.96:0/2454644257 >> 172.21.15.96:0/2454644257 conn(0x7f44700fbc70 msgr2=0x7f44700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- 172.21.15.96:0/2454644257 shutdown_connections 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- 172.21.15.96:0/2454644257 wait complete. 2024-03-18T15:46:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 Processor -- start 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.518+0000 7f4474f9d640 1 -- start start 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4470100700 0x7f44700699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f447010c380 con 0x7f44701022b0 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f447010c200 con 0x7f44701018f0 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f447010c500 con 0x7f4470100700 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4467fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45094/0 (socket says 172.21.15.96:45094) 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4467fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:36278/0 (socket says 172.21.15.96:36278) 2024-03-18T15:46:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 -- 172.21.15.96:0/2375543051 learned_addr learned my addr 172.21.15.96:0/2375543051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446e59c640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4470100700 0x7f44700699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 -- 172.21.15.96:0/2375543051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4470100700 msgr2=0x7f44700699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4470100700 0x7f44700699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 -- 172.21.15.96:0/2375543051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 msgr2=0x7f447019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 -- 172.21.15.96:0/2375543051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447019f6b0 con 0x7f44701022b0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4467fff640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f446ed9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4468037040 tx=0x7f4468032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f44677fe640 1 -- 172.21.15.96:0/2375543051 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468041020 con 0x7f44701022b0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f44677fe640 1 -- 172.21.15.96:0/2375543051 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4468034cd0 con 0x7f44701022b0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447019f8e0 con 0x7f44701022b0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f44677fe640 1 -- 172.21.15.96:0/2375543051 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4468039040 con 0x7f44701022b0 2024-03-18T15:46:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.519+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447019fdd0 con 0x7f44701022b0 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.521+0000 7f44677fe640 1 -- 172.21.15.96:0/2375543051 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4468044020 con 0x7f44701022b0 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f44677fe640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 0x7f445c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 Processor -- start 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f446e59c640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 0x7f445c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f44677fe640 1 -- 172.21.15.96:0/2375543051 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f44680f0930 con 0x7f44701022b0 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 -- start start 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4101b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41020830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc40f80a40 con 0x55fc4101b340 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc40f29920 con 0x55fc4101bd30 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.522+0000 7f4477b81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc40f7cae0 con 0x55fc41020ea0 2024-03-18T15:46:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4101b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:36858/0 (socket says 172.21.15.96:36858) 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446ed9d640 1 -- 172.21.15.96:0/2091440895 learned_addr learned my addr 172.21.15.96:0/2091440895 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f4467fff640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41020830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 0x7f445c0a4d60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4470101750 tx=0x7f4460005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 -- 172.21.15.96:0/2091440895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 msgr2=0x55fc41023330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 -- 172.21.15.96:0/2091440895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 msgr2=0x55fc41020830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41020830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 -- 172.21.15.96:0/2091440895 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc40f41460 con 0x55fc4101b340 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446ed9d640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f4467fff640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41020830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f446e59c640 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4101b760 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f446000e560 tx=0x7f4460014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406eb7dd48188592 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f4464ff9640 1 -- 172.21.15.96:0/2091440895 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4460026430 con 0x55fc4101b340 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f4464ff9640 1 -- 172.21.15.96:0/2091440895 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446001d8f0 con 0x55fc4101b340 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.523+0000 7f4464ff9640 1 -- 172.21.15.96:0/2091440895 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44600257d0 con 0x55fc4101b340 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- 172.21.15.96:0/2091440895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 msgr2=0x55fc4101b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4101b760 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f446000e560 tx=0x7f4460014000 comp rx=0 tx=0).stop 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- 172.21.15.96:0/2091440895 shutdown_connections 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41023330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41020830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 --2- 172.21.15.96:0/2091440895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4101b760 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- 172.21.15.96:0/2091440895 >> 172.21.15.96:0/2091440895 conn(0x55fc410168b0 msgr2=0x55fc41016cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- 172.21.15.96:0/2091440895 shutdown_connections 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- 172.21.15.96:0/2091440895 wait complete. 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.524+0000 7f4477b81740 1 -- start start 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 Processor -- start 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101b340 0x55fc41025660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41025ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc41023e50 con 0x55fc41020ea0 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fc40f3cd70 con 0x55fc4101bd30 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc41023fd0 con 0x55fc4101b340 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 10 client.0 _ll_register_callbacks cb 0x55fc40c94a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4467fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41025ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45114/0 (socket says 172.21.15.96:45114) 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446e59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101b340 0x55fc41025660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 -- 172.21.15.96:0/3247665945 learned_addr learned my addr 172.21.15.96:0/3247665945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 -- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101b340 msgr2=0x55fc41025660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101b340 0x55fc41025660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 -- 172.21.15.96:0/3247665945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 msgr2=0x55fc41025ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41025ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 -- 172.21.15.96:0/3247665945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc40c95210 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4467fff640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41025ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f446ed9d640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4468006010 tx=0x7f4468004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1269c45d72e345 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44680c5c90 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4468003970 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.525+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f446001d8f0 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f44680bda30 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 10 client.16956 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fc40d61620 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f446001d8f0 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f446803c030 con 0x55fc41020ea0 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 10 client.16956 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 20 client.16956 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41031580 0x55fc410339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 4 client.16956 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc40f8df80 con 0x55fc41031580 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.526+0000 7f446e59c640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41031580 0x55fc410339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f446e59c640 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41031580 0x55fc410339b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06937bbb5f2c706 server_cookie=7935d2d19471e9a2 in_seq=0 out_seq=0 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4455ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4455ffb640 1 -- 172.21.15.96:0/3247665945 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55fc40f8df80 con 0x55fc41031580 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4455ffb640 10 client.16956 handle_command_reply: tid=0 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4477b81740 1 client.16956 shutdown 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 msgr2=0x55fc4102cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4468006010 tx=0x7f4468004a40 comp rx=0 tx=0).stop 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 shutdown_connections 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.527+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41031580 0x55fc410339b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc41020ea0 0x55fc4102cbe0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101bd30 0x55fc41025ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4455ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 --2- 172.21.15.96:0/3247665945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101b340 0x55fc41025660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 >> 172.21.15.96:0/3247665945 conn(0x55fc410168b0 msgr2=0x55fc41015bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 shutdown_connections 2024-03-18T15:46:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 1 -- 172.21.15.96:0/3247665945 wait complete. 2024-03-18T15:46:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.528+0000 7f4477b81740 20 client.16956 trim_cache size 0 max 16384 2024-03-18T15:46:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 Processor -- start 2024-03-18T15:46:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 -- start start 2024-03-18T15:46:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 0x55fc41015320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41015860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a8c40 con 0x55fc4101bd30 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44701aa810 con 0x55fc4101b340 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4477b81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a9310 con 0x55fc41020ea0 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4467fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f446e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 0x55fc41015320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4467fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45116/0 (socket says 172.21.15.96:45116) 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f4467fff640 1 -- 172.21.15.96:0/2667929351 learned_addr learned my addr 172.21.15.96:0/2667929351 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.542+0000 7f446ed9d640 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41015860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 -- 172.21.15.96:0/2667929351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 msgr2=0x55fc41015860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41015860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 -- 172.21.15.96:0/2667929351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 msgr2=0x55fc41015320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 0x55fc41015320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 -- 172.21.15.96:0/2667929351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4101b720 con 0x55fc4101bd30 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f446e59c640 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 0x55fc41015320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4467fff640 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4458012ab0 tx=0x7f4458005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a719c8e27f43769 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4457fff640 1 -- 172.21.15.96:0/2667929351 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458022070 con 0x55fc4101bd30 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4457fff640 1 -- 172.21.15.96:0/2667929351 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445800f9e0 con 0x55fc4101bd30 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4457fff640 1 -- 172.21.15.96:0/2667929351 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4458003b50 con 0x55fc4101bd30 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 -- 172.21.15.96:0/2667929351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 msgr2=0x55fc4102cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4458012ab0 tx=0x7f4458005f90 comp rx=0 tx=0).stop 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 -- 172.21.15.96:0/2667929351 shutdown_connections 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc41020ea0 0x55fc41015860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101bd30 0x55fc4102cba0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 --2- 172.21.15.96:0/2667929351 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc4101b340 0x55fc41015320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 -- 172.21.15.96:0/2667929351 >> 172.21.15.96:0/2667929351 conn(0x55fc410168b0 msgr2=0x55fc4101d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 -- 172.21.15.96:0/2667929351 shutdown_connections 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.543+0000 7f4477b81740 1 -- 172.21.15.96:0/2667929351 wait complete. 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 -- start start 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 Processor -- start 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101bd30 0x55fc4102ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc41020ea0 0x55fc4102be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44701aa140 con 0x55fc4101b340 2024-03-18T15:46:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a9a70 con 0x55fc41020ea0 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44701a6c20 con 0x55fc4101bd30 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4477b81740 10 client.0 _ll_register_callbacks cb 0x7f44680b95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45132/0 (socket says 172.21.15.96:45132) 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 -- 172.21.15.96:0/740302577 learned_addr learned my addr 172.21.15.96:0/740302577 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f4467fff640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101bd30 0x55fc4102ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446ed9d640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc41020ea0 0x55fc4102be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 -- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101bd30 msgr2=0x55fc4102ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101bd30 0x55fc4102ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 -- 172.21.15.96:0/740302577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc41020ea0 msgr2=0x55fc4102be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc41020ea0 0x55fc4102be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 -- 172.21.15.96:0/740302577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4102c410 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.544+0000 7f446e59c640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f44600058c0 tx=0x7f446001d410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76a128eac2c0833 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446000e960 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f446002b040 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f445800f9e0 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f446000ed70 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 10 client.16962 fetch_fsmap learned FSMap version 34 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fc41097f60 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f445800f9e0 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f44600035f0 con 0x55fc4101b340 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 10 client.16962 fetch_fsmap finished waiting for FSMap version 34 2024-03-18T15:46:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 10 client.16962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 20 client.16962 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41098630 0x55fc4109aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 4 client.16962 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc40f882d0 con 0x55fc41098630 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.545+0000 7f4467fff640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41098630 0x55fc4109aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.546+0000 7f4467fff640 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41098630 0x55fc4109aa50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb411bb47228e5e server_cookie=b968726f8ac2ee1a in_seq=0 out_seq=0 2024-03-18T15:46:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.546+0000 7f4455ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4455ffb640 1 -- 172.21.15.96:0/740302577 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc40f882d0 con 0x55fc41098630 2024-03-18T15:46:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4455ffb640 10 client.16962 handle_command_reply: tid=0 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 client.16962 shutdown 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 msgr2=0x55fc4102a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f44600058c0 tx=0x7f446001d410 comp rx=0 tx=0).stop 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 shutdown_connections 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fc41020ea0 0x55fc4102be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc4101bd30 0x55fc4102ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55fc41098630 0x55fc4109aa50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 --2- 172.21.15.96:0/740302577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4101b340 0x55fc4102a920 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 >> 172.21.15.96:0/740302577 conn(0x55fc410168b0 msgr2=0x55fc41019b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4455ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 shutdown_connections 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 1 -- 172.21.15.96:0/740302577 wait complete. 2024-03-18T15:46:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.706+0000 7f4477b81740 20 client.16962 trim_cache size 0 max 16384 2024-03-18T15:46:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:46:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:46:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:46:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 msgr2=0x7f445c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 0x7f445c0a4d60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4470101750 tx=0x7f4460005f90 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 msgr2=0x7f447019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4468037040 tx=0x7f4468032f40 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 shutdown_connections 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f445c0a28a0 0x7f445c0a4d60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44701022b0 0x7f447019eda0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44701018f0 0x7f447019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 --2- 172.21.15.96:0/2375543051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4470100700 0x7f44700699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 >> 172.21.15.96:0/2375543051 conn(0x7f44700fbc70 msgr2=0x7f4470104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 shutdown_connections 2024-03-18T15:46:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:21.707+0000 7f4474f9d640 1 -- 172.21.15.96:0/2375543051 wait complete. 2024-03-18T15:46:22.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:22 smithi193 ceph-mon[36702]: pgmap v540: 97 pgs: 97 active+clean; 753 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-03-18T15:46:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:22 smithi096 ceph-mon[31930]: pgmap v540: 97 pgs: 97 active+clean; 753 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-03-18T15:46:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:22 smithi137 ceph-mon[35868]: pgmap v540: 97 pgs: 97 active+clean; 753 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.6 MiB/s wr, 576 op/s 2024-03-18T15:46:24.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:24 smithi137 ceph-mon[35868]: pgmap v541: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-18T15:46:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:24 smithi193 ceph-mon[36702]: pgmap v541: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-18T15:46:24.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:24 smithi096 ceph-mon[31930]: pgmap v541: 97 pgs: 97 active+clean; 756 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 648 op/s 2024-03-18T15:46:26.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:26 smithi193 ceph-mon[36702]: pgmap v542: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 752 op/s 2024-03-18T15:46:26.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:26 smithi096 ceph-mon[31930]: pgmap v542: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 752 op/s 2024-03-18T15:46:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:26 smithi137 ceph-mon[35868]: pgmap v542: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.1 MiB/s wr, 752 op/s 2024-03-18T15:46:27.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:27 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:27.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:27 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:28.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:27 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:28.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:28 smithi193 ceph-mon[36702]: pgmap v543: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 643 op/s 2024-03-18T15:46:28.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:28 smithi096 ceph-mon[31930]: pgmap v543: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 643 op/s 2024-03-18T15:46:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:28 smithi137 ceph-mon[35868]: pgmap v543: 97 pgs: 97 active+clean; 747 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 643 op/s 2024-03-18T15:46:30.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:30 smithi193 ceph-mon[36702]: pgmap v544: 97 pgs: 97 active+clean; 730 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 758 op/s 2024-03-18T15:46:30.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:30 smithi096 ceph-mon[31930]: pgmap v544: 97 pgs: 97 active+clean; 730 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 758 op/s 2024-03-18T15:46:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:30 smithi137 ceph-mon[35868]: pgmap v544: 97 pgs: 97 active+clean; 730 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 758 op/s 2024-03-18T15:46:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:32 smithi193 ceph-mon[36702]: pgmap v545: 97 pgs: 97 active+clean; 729 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-03-18T15:46:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:32 smithi096 ceph-mon[31930]: pgmap v545: 97 pgs: 97 active+clean; 729 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-03-18T15:46:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:32 smithi137 ceph-mon[35868]: pgmap v545: 97 pgs: 97 active+clean; 729 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 431 op/s 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 -- 172.21.15.96:0/3348118677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320107520 msgr2=0x7f3320109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 --2- 172.21.15.96:0/3348118677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320107520 0x7f3320109930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3310009900 tx=0x7f331002bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 -- 172.21.15.96:0/3348118677 shutdown_connections 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 --2- 172.21.15.96:0/3348118677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3320107520 0x7f3320109930 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 --2- 172.21.15.96:0/3348118677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 --2- 172.21.15.96:0/3348118677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33201022c0 0x7f33201046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 -- 172.21.15.96:0/3348118677 >> 172.21.15.96:0/3348118677 conn(0x7f33200fbc70 msgr2=0x7f33200fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 -- 172.21.15.96:0/3348118677 shutdown_connections 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 -- 172.21.15.96:0/3348118677 wait complete. 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.793+0000 7f332511b640 1 Processor -- start 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 -- start start 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3320107520 0x7f332019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f332010d480 con 0x7f33201022c0 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f332010d300 con 0x7f3320104bf0 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f332511b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f332010d600 con 0x7f3320107520 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43062/0 (socket says 172.21.15.96:43062) 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 -- 172.21.15.96:0/3608137756 learned_addr learned my addr 172.21.15.96:0/3608137756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331e59c640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331f59e640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3320107520 0x7f332019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 -- 172.21.15.96:0/3608137756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3320107520 msgr2=0x7f332019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3320107520 0x7f332019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 -- 172.21.15.96:0/3608137756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 msgr2=0x7f3320197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331ed9d640 1 -- 172.21.15.96:0/3608137756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332019f680 con 0x7f33201022c0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.794+0000 7f331e59c640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f331ed9d640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f331800ebc0 tx=0x7f331800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f3307fff640 1 -- 172.21.15.96:0/3608137756 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318010630 con 0x7f33201022c0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332019f970 con 0x7f33201022c0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f3307fff640 1 -- 172.21.15.96:0/3608137756 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3318010cb0 con 0x7f33201022c0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f3307fff640 1 -- 172.21.15.96:0/3608137756 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318018bd0 con 0x7f33201022c0 2024-03-18T15:46:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.795+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332019fec0 con 0x7f33201022c0 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3307fff640 1 -- 172.21.15.96:0/3608137756 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f33180107d0 con 0x7f33201022c0 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 Processor -- start 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 -- start start 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d0570760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3307fff640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 0x7f33080a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0575ea0 0x55e8d0578330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8d04d5a40 con 0x55e8d0575ea0 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8d047e920 con 0x55e8d0570d30 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.797+0000 7f3327de5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8d04d1ae0 con 0x55e8d0570340 2024-03-18T15:46:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331ed9d640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 0x7f33080a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0575ea0 0x55e8d0578330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54004/0 (socket says 172.21.15.96:54004) 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 -- 172.21.15.96:0/3592539346 learned_addr learned my addr 172.21.15.96:0/3592539346 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331f59e640 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d0570760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f3307fff640 1 -- 172.21.15.96:0/3608137756 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f33180cefd0 con 0x7f33201022c0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 -- 172.21.15.96:0/3592539346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 msgr2=0x55e8d0570760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d0570760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 -- 172.21.15.96:0/3592539346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0575ea0 msgr2=0x55e8d0578330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0575ea0 0x55e8d0578330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 -- 172.21.15.96:0/3592539346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8d0496460 con 0x55e8d0570d30 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331ed9d640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 0x7f33080a4d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f331800eb90 tx=0x7f331806d710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f331e59c640 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33140114f0 tx=0x7f33140119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46f82a7181cc15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f33057fa640 1 -- 172.21.15.96:0/3592539346 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314005020 con 0x55e8d0570d30 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f33057fa640 1 -- 172.21.15.96:0/3592539346 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33140047a0 con 0x55e8d0570d30 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.798+0000 7f33057fa640 1 -- 172.21.15.96:0/3592539346 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400f840 con 0x55e8d0570d30 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- 172.21.15.96:0/3592539346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 msgr2=0x55e8d0575830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f33140114f0 tx=0x7f33140119b0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- 172.21.15.96:0/3592539346 shutdown_connections 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0575ea0 0x55e8d0578330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570d30 0x55e8d0575830 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 --2- 172.21.15.96:0/3592539346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d0570760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- 172.21.15.96:0/3592539346 >> 172.21.15.96:0/3592539346 conn(0x55e8d056b8b0 msgr2=0x55e8d056bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- 172.21.15.96:0/3592539346 shutdown_connections 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- 172.21.15.96:0/3592539346 wait complete. 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 -- start start 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.799+0000 7f3327de5740 1 Processor -- start 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570340 0x55e8d057a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0575ea0 0x55e8d0581b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e8d0578e50 con 0x55e8d0570d30 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8d0491d70 con 0x55e8d0570340 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8d0578fd0 con 0x55e8d0575ea0 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 10 client.0 _ll_register_callbacks cb 0x55e8d01e9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570340 0x55e8d057a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43078/0 (socket says 172.21.15.96:43078) 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 -- 172.21.15.96:0/2121330264 learned_addr learned my addr 172.21.15.96:0/2121330264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331f59e640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0575ea0 0x55e8d0581b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 -- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0575ea0 msgr2=0x55e8d0581b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0575ea0 0x55e8d0581b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 -- 172.21.15.96:0/2121330264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570340 msgr2=0x55e8d057a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570340 0x55e8d057a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 -- 172.21.15.96:0/2121330264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8d01ea210 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f331e59c640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f331401b040 tx=0x7f3314005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee8c9950e753adb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3302ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314028070 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3314004020 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.800+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33140047a0 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3314033050 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 10 client.16974 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e8d02b6620 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33140047a0 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f331400f5a0 con 0x55e8d0570d30 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 10 client.16974 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 20 client.16974 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d0586220 0x55e8d0588610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 4 client.16974 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8d04e2f80 con 0x55e8d0586220 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.801+0000 7f331f59e640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d0586220 0x55e8d0588610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f331f59e640 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d0586220 0x55e8d0588610 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f303ae6b2159d6b server_cookie=c317816b6c088288 in_seq=0 out_seq=0 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3302ff5640 10 client.16974 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3302ff5640 1 -- 172.21.15.96:0/2121330264 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e8d04e2f80 con 0x55e8d0586220 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3302ff5640 10 client.16974 handle_command_reply: tid=0 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 client.16974 shutdown 2024-03-18T15:46:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 msgr2=0x55e8d057ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f331401b040 tx=0x7f3314005240 comp rx=0 tx=0).stop 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 shutdown_connections 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0575ea0 0x55e8d0581b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d0586220 0x55e8d0588610 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d057ab00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3302ff5640 0 client.16974 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 --2- 172.21.15.96:0/2121330264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0570340 0x55e8d057a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 >> 172.21.15.96:0/2121330264 conn(0x55e8d056b8b0 msgr2=0x55e8d056abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 shutdown_connections 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.802+0000 7f3327de5740 1 -- 172.21.15.96:0/2121330264 wait complete. 2024-03-18T15:46:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.803+0000 7f3327de5740 20 client.16974 trim_cache size 0 max 16384 2024-03-18T15:46:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.817+0000 7f3327de5740 1 Processor -- start 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.817+0000 7f3327de5740 1 -- start start 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d056a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d056ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33201a8d30 con 0x55e8d0570d30 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33201aa930 con 0x55e8d0575ea0 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3327de5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33201a9430 con 0x55e8d0570340 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43080/0 (socket says 172.21.15.96:43080) 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 -- 172.21.15.96:0/644537911 learned_addr learned my addr 172.21.15.96:0/644537911 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331f59e640 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d056a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331f59e640 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d056ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 -- 172.21.15.96:0/644537911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 msgr2=0x55e8d056a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d056a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 -- 172.21.15.96:0/644537911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 msgr2=0x55e8d056ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d056ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 -- 172.21.15.96:0/644537911 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8d0588940 con 0x55e8d0570d30 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f331e59c640 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3314011eb0 tx=0x7f3314009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f8edb0d39a1281 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3304ff9640 1 -- 172.21.15.96:0/644537911 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314028040 con 0x55e8d0570d30 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3304ff9640 1 -- 172.21.15.96:0/644537911 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3314004c60 con 0x55e8d0570d30 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.818+0000 7f3304ff9640 1 -- 172.21.15.96:0/644537911 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331400f700 con 0x55e8d0570d30 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- 172.21.15.96:0/644537911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 msgr2=0x55e8d056a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3314011eb0 tx=0x7f3314009f20 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- 172.21.15.96:0/644537911 shutdown_connections 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d056ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d056a860 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- 172.21.15.96:0/644537911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d056a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- 172.21.15.96:0/644537911 >> 172.21.15.96:0/644537911 conn(0x55e8d056b8b0 msgr2=0x55e8d056bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- 172.21.15.96:0/644537911 shutdown_connections 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- 172.21.15.96:0/644537911 wait complete. 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- start start 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 Processor -- start 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d05e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d05ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33201aa260 con 0x55e8d0570d30 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33201a9b90 con 0x55e8d0575ea0 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33201a6d10 con 0x55e8d0570340 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f3327de5740 10 client.0 _ll_register_callbacks cb 0x7f33180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.819+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d05e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43094/0 (socket says 172.21.15.96:43094) 2024-03-18T15:46:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 -- 172.21.15.96:0/2426746812 learned_addr learned my addr 172.21.15.96:0/2426746812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331f59e640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d05ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 -- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 msgr2=0x55e8d05e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d05e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 -- 172.21.15.96:0/2426746812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 msgr2=0x55e8d05ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d05ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 -- 172.21.15.96:0/2426746812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8d05efbc0 con 0x55e8d0570d30 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331f59e640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d05e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f331e59c640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3314009300 tx=0x7f3314004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422988d36106d074 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3302ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314028040 con 0x55e8d0570d30 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f331400f6e0 con 0x55e8d0570d30 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314004c60 con 0x55e8d0570d30 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f331400fd70 con 0x55e8d0570d30 2024-03-18T15:46:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3327de5740 10 client.16980 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e8d05f0ed0 con 0x55e8d0570d30 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.820+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3314004c60 con 0x55e8d0570d30 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f331403f030 con 0x55e8d0570d30 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 10 client.16980 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 20 client.16980 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d05f18c0 0x55e8d05f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 4 client.16980 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8d04dd2d0 con 0x55e8d05f18c0 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f331f59e640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d05f18c0 0x55e8d05f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f3302ff5640 10 client.16980 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.821+0000 7f331f59e640 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d05f18c0 0x55e8d05f3ce0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26983f76907ca6a4 server_cookie=45d8c1221b7f955 in_seq=0 out_seq=0 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3302ff5640 1 -- 172.21.15.96:0/2426746812 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8d04dd2d0 con 0x55e8d05f18c0 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3302ff5640 10 client.16980 handle_command_reply: tid=0 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 client.16980 shutdown 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 msgr2=0x55e8d05e8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3314009300 tx=0x7f3314004ea0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 shutdown_connections 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e8d0575ea0 0x55e8d05ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e8d05f18c0 0x55e8d05f3ce0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e8d0570d30 0x55e8d05e8590 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3302ff5640 0 client.16980 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 --2- 172.21.15.96:0/2426746812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e8d0570340 0x55e8d05e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.837+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 >> 172.21.15.96:0/2426746812 conn(0x55e8d056b8b0 msgr2=0x55e8d056bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 shutdown_connections 2024-03-18T15:46:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f3327de5740 1 -- 172.21.15.96:0/2426746812 wait complete. 2024-03-18T15:46:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f3327de5740 20 client.16980 trim_cache size 0 max 16384 2024-03-18T15:46:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:46:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:46:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:46:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 msgr2=0x7f33080a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 0x7f33080a4d60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f331800eb90 tx=0x7f331806d710 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 msgr2=0x7f3320197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f331800ebc0 tx=0x7f331800c6a0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 shutdown_connections 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f33080a28a0 0x7f33080a4d60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3320107520 0x7f332019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3320104bf0 0x7f3320197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 --2- 172.21.15.96:0/3608137756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f33201022c0 0x7f3320197a60 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 >> 172.21.15.96:0/3608137756 conn(0x7f33200fbc70 msgr2=0x7f3320102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 shutdown_connections 2024-03-18T15:46:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:34.838+0000 7f332511b640 1 -- 172.21.15.96:0/3608137756 wait complete. 2024-03-18T15:46:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:34 smithi193 ceph-mon[36702]: pgmap v546: 97 pgs: 97 active+clean; 712 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-03-18T15:46:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:34 smithi096 ceph-mon[31930]: pgmap v546: 97 pgs: 97 active+clean; 712 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-03-18T15:46:35.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:34 smithi137 ceph-mon[35868]: pgmap v546: 97 pgs: 97 active+clean; 712 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 450 op/s 2024-03-18T15:46:35.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:35 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:46:36.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:35 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:46:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:35 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:46:36.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:46:36.268 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:46:36.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 -- 172.21.15.96:0/1753867688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 msgr2=0x7f3b10108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 --2- 172.21.15.96:0/1753867688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b10108b20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009ca0 tx=0x7f3b0802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 -- 172.21.15.96:0/1753867688 shutdown_connections 2024-03-18T15:46:36.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 --2- 172.21.15.96:0/1753867688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b10108b20 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 --2- 172.21.15.96:0/1753867688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 0x7f3b10101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 --2- 172.21.15.96:0/1753867688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 -- 172.21.15.96:0/1753867688 >> 172.21.15.96:0/1753867688 conn(0x7f3b100fbc70 msgr2=0x7f3b100fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 -- 172.21.15.96:0/1753867688 shutdown_connections 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.376+0000 7f3b1508c640 1 -- 172.21.15.96:0/1753867688 wait complete. 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 Processor -- start 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 -- start start 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 0x7f3b10198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c360 con 0x7f3b10102290 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c1e0 con 0x7f3b101018d0 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.377+0000 7f3b1508c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1010c4e0 con 0x7f3b101006e0 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43108/0 (socket says 172.21.15.96:43108) 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 -- 172.21.15.96:0/2029513951 learned_addr learned my addr 172.21.15.96:0/2029513951 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 0x7f3b10198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 -- 172.21.15.96:0/2029513951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 msgr2=0x7f3b10197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 -- 172.21.15.96:0/2029513951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 msgr2=0x7f3b10198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 0x7f3b10198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 -- 172.21.15.96:0/2029513951 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1019f720 con 0x7f3b10102290 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3b08037040 tx=0x7f3b08032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3af7fff640 1 -- 172.21.15.96:0/2029513951 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08041020 con 0x7f3b10102290 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3af7fff640 1 -- 172.21.15.96:0/2029513951 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b08034cd0 con 0x7f3b10102290 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1019f9b0 con 0x7f3b10102290 2024-03-18T15:46:36.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3af7fff640 1 -- 172.21.15.96:0/2029513951 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08039040 con 0x7f3b10102290 2024-03-18T15:46:36.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.378+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1019fea0 con 0x7f3b10102290 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.380+0000 7f3af7fff640 1 -- 172.21.15.96:0/2029513951 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f3b08044020 con 0x7f3b10102290 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 Processor -- start 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 -- start start 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b42a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3af7fff640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 0x7f3af80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b432160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d4b38f500 con 0x563d4b42ab60 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d4b337be0 con 0x563d4b42a170 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d4b38b5a0 con 0x563d4b42fcd0 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 0x7f3af80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43116/0 (socket says 172.21.15.96:43116) 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0f59e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b432160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0e59c640 1 -- 172.21.15.96:0/196289915 learned_addr learned my addr 172.21.15.96:0/196289915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3af7fff640 1 -- 172.21.15.96:0/2029513951 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3b080f0930 con 0x7f3b10102290 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b42a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0f59e640 1 -- 172.21.15.96:0/196289915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 msgr2=0x563d4b42a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.381+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b42a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0f59e640 1 -- 172.21.15.96:0/196289915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 msgr2=0x563d4b42f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0f59e640 1 -- 172.21.15.96:0/196289915 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4b34b4b0 con 0x563d4b42fcd0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 0x7f3af80a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3b10101730 tx=0x7f3b00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b432160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b080b9790 tx=0x7f3b0803a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=540007bf4bfa989e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3af57fa640 1 -- 172.21.15.96:0/196289915 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080b95d0 con 0x563d4b42fcd0 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3af57fa640 1 -- 172.21.15.96:0/196289915 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b080c6900 con 0x563d4b42fcd0 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.382+0000 7f3af57fa640 1 -- 172.21.15.96:0/196289915 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080ca640 con 0x563d4b42fcd0 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- 172.21.15.96:0/196289915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 msgr2=0x563d4b432160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b432160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b080b9790 tx=0x7f3b0803a8e0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- 172.21.15.96:0/196289915 shutdown_connections 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b432160 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b42f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/196289915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b42a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- 172.21.15.96:0/196289915 >> 172.21.15.96:0/196289915 conn(0x563d4b4256e0 msgr2=0x563d4b425ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- 172.21.15.96:0/196289915 shutdown_connections 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- 172.21.15.96:0/196289915 wait complete. 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- start start 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 Processor -- start 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b434310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d4b432a40 con 0x563d4b42ab60 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563d4b34b160 con 0x563d4b42a170 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d4b432bc0 con 0x563d4b42fcd0 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b17c7e740 10 client.0 _ll_register_callbacks cb 0x563d4b32f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b434310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43130/0 (socket says 172.21.15.96:43130) 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b434310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54062/0 (socket says 172.21.15.96:54062) 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0e59c640 1 -- 172.21.15.96:0/430344387 learned_addr learned my addr 172.21.15.96:0/430344387 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.383+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 -- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 msgr2=0x563d4b43b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 -- 172.21.15.96:0/430344387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 msgr2=0x563d4b434310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b434310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 -- 172.21.15.96:0/430344387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4b16d7d0 con 0x563d4b42ab60 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3b04012ab0 tx=0x7f3b04009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefa71883c813094 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0400ebc0 con 0x563d4b42ab60 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3b0400ed60 con 0x563d4b42ab60 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b080c6900 con 0x563d4b42ab60 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b04024030 con 0x563d4b42ab60 2024-03-18T15:46:36.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b17c7e740 10 client.16992 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563d4b0a33f0 con 0x563d4b42ab60 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b080c6900 con 0x563d4b42ab60 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.384+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3b0401d030 con 0x563d4b42ab60 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 10 client.16992 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 20 client.16992 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b43fe30 0x563d4b4422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 4 client.16992 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d4b39ca40 con 0x563d4b43fe30 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b43fe30 0x563d4b4422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3af2ff5640 10 client.16992 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.385+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b43fe30 0x563d4b4422d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc100c1492ea126 server_cookie=61ee165ca44578ef in_seq=0 out_seq=0 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3af2ff5640 1 -- 172.21.15.96:0/430344387 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563d4b39ca40 con 0x563d4b43fe30 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3af2ff5640 10 client.16992 handle_command_reply: tid=0 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 client.16992 shutdown 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 msgr2=0x563d4b434850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3b04012ab0 tx=0x7f3b04009f50 comp rx=0 tx=0).stop 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 shutdown_connections 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b43fe30 0x563d4b4422d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b434850 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/430344387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b434310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3af2ff5640 0 client.16992 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.386+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 >> 172.21.15.96:0/430344387 conn(0x563d4b4256e0 msgr2=0x563d4b424900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.387+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 shutdown_connections 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.387+0000 7f3b17c7e740 1 -- 172.21.15.96:0/430344387 wait complete. 2024-03-18T15:46:36.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.387+0000 7f3b17c7e740 20 client.16992 trim_cache size 0 max 16384 2024-03-18T15:46:36.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.414+0000 7f3b17c7e740 1 Processor -- start 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.414+0000 7f3b17c7e740 1 -- start start 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42ab60 0x563d4b43b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b4245b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101a8d10 con 0x563d4b42a170 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101aa910 con 0x563d4b42ab60 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.415+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101a9410 con 0x563d4b42fcd0 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.417+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.417+0000 7f3b0ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43138/0 (socket says 172.21.15.96:43138) 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.417+0000 7f3b0ed9d640 1 -- 172.21.15.96:0/3040691800 learned_addr learned my addr 172.21.15.96:0/3040691800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.420+0000 7f3b0f59e640 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b4245b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.420+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42ab60 0x563d4b43b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.421+0000 7f3b0ed9d640 1 -- 172.21.15.96:0/3040691800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 msgr2=0x563d4b4245b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.421+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b4245b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.421+0000 7f3b0ed9d640 1 -- 172.21.15.96:0/3040691800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42ab60 msgr2=0x563d4b43b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.421+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42ab60 0x563d4b43b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.421+0000 7f3b0ed9d640 1 -- 172.21.15.96:0/3040691800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4b0630f0 con 0x563d4b42a170 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.422+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x563d4b426650 tx=0x7f3b00014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441b35c49c6264c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.422+0000 7f3af4ff9640 1 -- 172.21.15.96:0/3040691800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b00051070 con 0x563d4b42a170 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.422+0000 7f3af4ff9640 1 -- 172.21.15.96:0/3040691800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0000fe70 con 0x563d4b42a170 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.423+0000 7f3af4ff9640 1 -- 172.21.15.96:0/3040691800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b000278b0 con 0x563d4b42a170 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.423+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3040691800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 msgr2=0x563d4b424070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.423+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x563d4b426650 tx=0x7f3b00014000 comp rx=0 tx=0).stop 2024-03-18T15:46:36.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.424+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3040691800 shutdown_connections 2024-03-18T15:46:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.424+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b4245b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.424+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42ab60 0x563d4b43b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.424+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3040691800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42a170 0x563d4b424070 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.424+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3040691800 >> 172.21.15.96:0/3040691800 conn(0x563d4b4256e0 msgr2=0x563d4b42bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.426+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3040691800 shutdown_connections 2024-03-18T15:46:36.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.426+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3040691800 wait complete. 2024-03-18T15:46:36.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.426+0000 7f3b17c7e740 1 -- start start 2024-03-18T15:46:36.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.426+0000 7f3b17c7e740 1 Processor -- start 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b438f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101aa240 con 0x563d4b42ab60 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101a9b70 con 0x563d4b42a170 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b101a6cf0 con 0x563d4b42fcd0 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b17c7e740 10 client.0 _ll_register_callbacks cb 0x7f3b040129c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43152/0 (socket says 172.21.15.96:43152) 2024-03-18T15:46:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 -- 172.21.15.96:0/3701370369 learned_addr learned my addr 172.21.15.96:0/3701370369 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b438f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 -- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 msgr2=0x563d4b43a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 -- 172.21.15.96:0/3701370369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 msgr2=0x563d4b438f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.427+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b438f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3b0e59c640 1 -- 172.21.15.96:0/3701370369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4b3569d0 con 0x563d4b42ab60 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3b0e59c640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009800 tx=0x7f3b04011270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94a06e350eee174 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3af2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b04012240 con 0x563d4b42ab60 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b0000fe70 con 0x563d4b42ab60 2024-03-18T15:46:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.428+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3b040123e0 con 0x563d4b42ab60 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b04004040 con 0x563d4b42ab60 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 10 client.16998 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563d4b43bb70 con 0x563d4b42ab60 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0000fe70 con 0x563d4b42ab60 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3b0401d070 con 0x563d4b42ab60 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 10 client.16998 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 20 client.16998 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b4af220 0x563d4b4b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 4 client.16998 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:46:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d4b396d90 con 0x563d4b4af220 2024-03-18T15:46:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.429+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b4af220 0x563d4b4b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.430+0000 7f3af2ff5640 10 client.16998 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.430+0000 7f3b0ed9d640 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b4af220 0x563d4b4b1640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89da57f6945d619 server_cookie=61c0e1edf55f83aa in_seq=0 out_seq=0 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3af2ff5640 1 -- 172.21.15.96:0/3701370369 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d4b396d90 con 0x563d4b4af220 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3af2ff5640 10 client.16998 handle_command_reply: tid=0 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 client.16998 shutdown 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 msgr2=0x563d4b4394c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009800 tx=0x7f3b04011270 comp rx=0 tx=0).stop 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 shutdown_connections 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d4b42fcd0 0x563d4b43a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563d4b4af220 0x563d4b4b1640 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d4b42ab60 0x563d4b4394c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 --2- 172.21.15.96:0/3701370369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563d4b42a170 0x563d4b438f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 >> 172.21.15.96:0/3701370369 conn(0x563d4b4256e0 msgr2=0x563d4b428940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3af2ff5640 0 client.16998 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 shutdown_connections 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.431+0000 7f3b17c7e740 1 -- 172.21.15.96:0/3701370369 wait complete. 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b17c7e740 20 client.16998 trim_cache size 0 max 16384 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:46:36.434 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:46:36.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 msgr2=0x7f3af80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 0x7f3af80a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3b10101730 tx=0x7f3b00005f90 comp rx=0 tx=0).stop 2024-03-18T15:46:36.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 msgr2=0x7f3b1019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3b08037040 tx=0x7f3b08032f40 comp rx=0 tx=0).stop 2024-03-18T15:46:36.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 shutdown_connections 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3af80a2800 0x7f3af80a4cc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b10102290 0x7f3b1019f0b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b101018d0 0x7f3b10198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 --2- 172.21.15.96:0/2029513951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b101006e0 0x7f3b10197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.432+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 >> 172.21.15.96:0/2029513951 conn(0x7f3b100fbc70 msgr2=0x7f3b100fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.433+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 shutdown_connections 2024-03-18T15:46:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.433+0000 7f3b1508c640 1 -- 172.21.15.96:0/2029513951 wait complete. 2024-03-18T15:46:36.447 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:46:36.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:46:36.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 -- 172.21.15.96:0/3211439967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 msgr2=0x7ff610108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 --2- 172.21.15.96:0/3211439967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff610108b40 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff608009ca0 tx=0x7ff60802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 -- 172.21.15.96:0/3211439967 shutdown_connections 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 --2- 172.21.15.96:0/3211439967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff610108b40 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 --2- 172.21.15.96:0/3211439967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 0x7ff610101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 --2- 172.21.15.96:0/3211439967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 0x7ff610100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.562+0000 7ff6166f1640 1 -- 172.21.15.96:0/3211439967 >> 172.21.15.96:0/3211439967 conn(0x7ff6100fbc70 msgr2=0x7ff6100fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- 172.21.15.96:0/3211439967 shutdown_connections 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- 172.21.15.96:0/3211439967 wait complete. 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 Processor -- start 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- start start 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 0x7ff610197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 0x7ff610198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff61010c360 con 0x7ff6101022b0 2024-03-18T15:46:36.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff61010c1e0 con 0x7ff6101018f0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff6166f1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff61010c4e0 con 0x7ff610100700 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.563+0000 7ff614c8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff60ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 0x7ff610197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43164/0 (socket says 172.21.15.96:43164) 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 -- 172.21.15.96:0/4048772731 learned_addr learned my addr 172.21.15.96:0/4048772731 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff60f7fe640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 0x7ff610198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 -- 172.21.15.96:0/4048772731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 msgr2=0x7ff610197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 0x7ff610197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 -- 172.21.15.96:0/4048772731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 msgr2=0x7ff610198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 0x7ff610198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 -- 172.21.15.96:0/4048772731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61019f710 con 0x7ff6101022b0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff614c8e640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff60802c630 tx=0x7ff608032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608041020 con 0x7ff6101022b0 2024-03-18T15:46:36.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff608034cd0 con 0x7ff6101022b0 2024-03-18T15:46:36.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61019f9a0 con 0x7ff6101022b0 2024-03-18T15:46:36.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff608039040 con 0x7ff6101022b0 2024-03-18T15:46:36.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.564+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61019fe90 con 0x7ff6101022b0 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.565+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6101a00f0 con 0x7ff6101022b0 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.566+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff608044020 con 0x7ff6101022b0 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.567+0000 7ff60d7fa640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 0x7ff5f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.567+0000 7ff60ffff640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 0x7ff5f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.567+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7ff6080f0cf0 con 0x7ff6101022b0 2024-03-18T15:46:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.567+0000 7ff60ffff640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 0x7ff5f80a4cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff610101750 tx=0x7ff600005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.571+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7ff60804b020 con 0x7ff6101022b0 2024-03-18T15:46:36.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.796+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6101a0480 con 0x7ff6101022b0 2024-03-18T15:46:36.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.796+0000 7ff60d7fa640 1 -- 172.21.15.96:0/4048772731 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11941 (secure 0 0 0) 0x7ff6080bddf0 con 0x7ff6101022b0 2024-03-18T15:46:36.797 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:46:36.797 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:46:26.570089+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/3504042144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":3504042144},{"type":"v1","addr":"172.21.15.137:6835","nonce":3504042144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24472,24493,24485,24452,24460]},"id":1}]} 2024-03-18T15:46:36.798 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-03-18T15:46:36.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 msgr2=0x7ff5f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 0x7ff5f80a4cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff610101750 tx=0x7ff600005f90 comp rx=0 tx=0).stop 2024-03-18T15:46:36.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 msgr2=0x7ff61019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff60802c630 tx=0x7ff608032ea0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 shutdown_connections 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff5f80a2800 0x7ff5f80a4cc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6101022b0 0x7ff61019f0a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6101018f0 0x7ff610198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 --2- 172.21.15.96:0/4048772731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff610100700 0x7ff610197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 >> 172.21.15.96:0/4048772731 conn(0x7ff6100fbc70 msgr2=0x7ff6100fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 shutdown_connections 2024-03-18T15:46:36.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.798+0000 7ff6166f1640 1 -- 172.21.15.96:0/4048772731 wait complete. 2024-03-18T15:46:36.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 -- 172.21.15.96:0/1708211357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 msgr2=0x7f64401002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 --2- 172.21.15.96:0/1708211357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f64401002b0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f643c009900 tx=0x7f643c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 -- 172.21.15.96:0/1708211357 shutdown_connections 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 --2- 172.21.15.96:0/1708211357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 0x7f64401082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 --2- 172.21.15.96:0/1708211357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 0x7f6440101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 --2- 172.21.15.96:0/1708211357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f64401002b0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 -- 172.21.15.96:0/1708211357 >> 172.21.15.96:0/1708211357 conn(0x7f64400fb3e0 msgr2=0x7f64400fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 -- 172.21.15.96:0/1708211357 shutdown_connections 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.916+0000 7f644844b640 1 -- 172.21.15.96:0/1708211357 wait complete. 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 Processor -- start 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 -- start start 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 0x7f6440198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 0x7f644019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f644010d530 con 0x7f64400ffe90 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f644010d3b0 con 0x7f6440100fe0 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f644844b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f644010d6b0 con 0x7f64401019a0 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43176/0 (socket says 172.21.15.96:43176) 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 -- 172.21.15.96:0/1409279557 learned_addr learned my addr 172.21.15.96:0/1409279557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64459e6640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 0x7f6440198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64469e8640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 0x7f644019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 -- 172.21.15.96:0/1409279557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 msgr2=0x7f644019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 0x7f644019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 -- 172.21.15.96:0/1409279557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 msgr2=0x7f6440198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 0x7f6440198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 -- 172.21.15.96:0/1409279557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019f710 con 0x7f64400ffe90 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64461e7640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f643c000c00 tx=0x7f643c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.917+0000 7f64377fe640 1 -- 172.21.15.96:0/1409279557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c043020 con 0x7f64400ffe90 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.918+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019f9a0 con 0x7f64400ffe90 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.918+0000 7f64377fe640 1 -- 172.21.15.96:0/1409279557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c0054e0 con 0x7f64400ffe90 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.918+0000 7f64377fe640 1 -- 172.21.15.96:0/1409279557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c003c40 con 0x7f64400ffe90 2024-03-18T15:46:36.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.918+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644019ff40 con 0x7f64400ffe90 2024-03-18T15:46:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.920+0000 7f64377fe640 1 -- 172.21.15.96:0/1409279557 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f643c046020 con 0x7f64400ffe90 2024-03-18T15:46:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.920+0000 7f644b036740 1 Processor -- start 2024-03-18T15:46:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 -- start start 2024-03-18T15:46:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd588c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd58dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd4ed490 con 0x55d7fd589240 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd495c90 con 0x55d7fd58e3b0 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f644b036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd4e9530 con 0x55d7fd588850 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64377fe640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 0x7f642c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd58dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64469e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd588c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54112/0 (socket says 172.21.15.96:54112) 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64469e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd588c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33060/0 (socket says 172.21.15.96:33060) 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 -- 172.21.15.96:0/949394362 learned_addr learned my addr 172.21.15.96:0/949394362 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64469e8640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 0x7f642c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64377fe640 1 -- 172.21.15.96:0/1409279557 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f643c0f0b00 con 0x7f64400ffe90 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 -- 172.21.15.96:0/949394362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 msgr2=0x55d7fd588c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd588c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 -- 172.21.15.96:0/949394362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 msgr2=0x55d7fd58dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd58dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 -- 172.21.15.96:0/949394362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fd550500 con 0x55d7fd58e3b0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64469e8640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 0x7f642c0a4cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55d7fd589aa0 tx=0x7f6438009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.921+0000 7f64459e6640 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6430015c60 tx=0x7f6430009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c008850be24653 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f6434bfe640 1 -- 172.21.15.96:0/949394362 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430004220 con 0x55d7fd58e3b0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f6434bfe640 1 -- 172.21.15.96:0/949394362 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643000fe80 con 0x55d7fd58e3b0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f6434bfe640 1 -- 172.21.15.96:0/949394362 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64300047a0 con 0x55d7fd58e3b0 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 -- 172.21.15.96:0/949394362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 msgr2=0x55d7fd590840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6430015c60 tx=0x7f6430009c20 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 -- 172.21.15.96:0/949394362 shutdown_connections 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd590840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd58dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 --2- 172.21.15.96:0/949394362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd588c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.922+0000 7f644b036740 1 -- 172.21.15.96:0/949394362 >> 172.21.15.96:0/949394362 conn(0x55d7fd583dc0 msgr2=0x55d7fd5841c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- 172.21.15.96:0/949394362 shutdown_connections 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- 172.21.15.96:0/949394362 wait complete. 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- start start 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 Processor -- start 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd589240 0x55d7fd5932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd59a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd591360 con 0x55d7fd588850 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd4a9230 con 0x55d7fd58e3b0 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7fd5914e0 con 0x55d7fd589240 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f644b036740 10 client.0 _ll_register_callbacks cb 0x55d7fd4e5010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:36.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43180/0 (socket says 172.21.15.96:43180) 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64461e7640 1 -- 172.21.15.96:0/3221055304 learned_addr learned my addr 172.21.15.96:0/3221055304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64459e6640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd589240 0x55d7fd5932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64469e8640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd59a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.923+0000 7f64461e7640 1 -- 172.21.15.96:0/3221055304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd589240 msgr2=0x55d7fd5932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f64461e7640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd589240 0x55d7fd5932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f64461e7640 1 -- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 msgr2=0x55d7fd59a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f64461e7640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd59a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f64461e7640 1 -- 172.21.15.96:0/3221055304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fd4b8100 con 0x55d7fd588850 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f64461e7640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f643c02c170 tx=0x7f643c08f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8271ca946f909c20 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0bcdd0 con 0x55d7fd588850 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f643c0bb770 con 0x55d7fd588850 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7fd4fa9d0 con 0x55d7fd588850 2024-03-18T15:46:36.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f643c004250 con 0x55d7fd588850 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 10 client.17016 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d7fd2013f0 con 0x55d7fd588850 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d7fd4fa9d0 con 0x55d7fd588850 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f643c0c1150 con 0x55d7fd588850 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 10 client.17016 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 10 client.17016 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 20 client.17016 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd59ebd0 0x55d7fd5a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 4 client.17016 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.924+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7fd4f4d20 con 0x55d7fd59ebd0 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.925+0000 7f64459e6640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd59ebd0 0x55d7fd5a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.925+0000 7f6429ffb640 10 client.17016 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.925+0000 7f64459e6640 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd59ebd0 0x55d7fd5a1040 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a98215e090ceef75 server_cookie=8b4347692effb9be in_seq=0 out_seq=0 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f6429ffb640 1 -- 172.21.15.96:0/3221055304 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d7fd4f4d20 con 0x55d7fd59ebd0 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f6429ffb640 10 client.17016 handle_command_reply: tid=0 2024-03-18T15:46:36.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 client.17016 shutdown 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 msgr2=0x55d7fd592d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f643c02c170 tx=0x7f643c08f0f0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 shutdown_connections 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd59ebd0 0x55d7fd5a1040 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd59a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f6429ffb640 0 client.17016 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd589240 0x55d7fd5932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 --2- 172.21.15.96:0/3221055304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd592d90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 >> 172.21.15.96:0/3221055304 conn(0x55d7fd583dc0 msgr2=0x55d7fd582fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 shutdown_connections 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 1 -- 172.21.15.96:0/3221055304 wait complete. 2024-03-18T15:46:36.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.926+0000 7f644b036740 20 client.17016 trim_cache size 0 max 16384 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: pgmap v547: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 880 KiB/s wr, 448 op/s 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:46:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:46:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.940+0000 7f644b036740 1 Processor -- start 2024-03-18T15:46:36.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.940+0000 7f644b036740 1 -- start start 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd589240 0x55d7fd582750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd58e3b0 0x55d7fd582c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a6b70 con 0x55d7fd588850 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a8770 con 0x55d7fd589240 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f644b036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a7270 con 0x55d7fd58e3b0 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43188/0 (socket says 172.21.15.96:43188) 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 -- 172.21.15.96:0/1020660661 learned_addr learned my addr 172.21.15.96:0/1020660661 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64459e6640 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd589240 0x55d7fd582750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64469e8640 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd58e3b0 0x55d7fd582c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 -- 172.21.15.96:0/1020660661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd58e3b0 msgr2=0x55d7fd582c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd58e3b0 0x55d7fd582c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 -- 172.21.15.96:0/1020660661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd589240 msgr2=0x55d7fd582750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd589240 0x55d7fd582750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 -- 172.21.15.96:0/1020660661 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fd2cb7d0 con 0x55d7fd588850 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f64461e7640 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f643c0b98d0 tx=0x7f643c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72df3e15714eb0ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f642bfff640 1 -- 172.21.15.96:0/1020660661 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0d5070 con 0x55d7fd588850 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f642bfff640 1 -- 172.21.15.96:0/1020660661 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c0035f0 con 0x55d7fd588850 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.941+0000 7f642bfff640 1 -- 172.21.15.96:0/1020660661 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c004c60 con 0x55d7fd588850 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- 172.21.15.96:0/1020660661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 msgr2=0x55d7fd593010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f643c0b98d0 tx=0x7f643c005e50 comp rx=0 tx=0).stop 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- 172.21.15.96:0/1020660661 shutdown_connections 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd58e3b0 0x55d7fd582c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd589240 0x55d7fd582750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 --2- 172.21.15.96:0/1020660661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd588850 0x55d7fd593010 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- 172.21.15.96:0/1020660661 >> 172.21.15.96:0/1020660661 conn(0x55d7fd583dc0 msgr2=0x55d7fd5878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- 172.21.15.96:0/1020660661 shutdown_connections 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- 172.21.15.96:0/1020660661 wait complete. 2024-03-18T15:46:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 -- start start 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.942+0000 7f644b036740 1 Processor -- start 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd597700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd598c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a80a0 con 0x55d7fd589240 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a79d0 con 0x55d7fd58e3b0 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f64401a4b50 con 0x55d7fd588850 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 10 client.0 _ll_register_callbacks cb 0x7f643c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43192/0 (socket says 172.21.15.96:43192) 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 -- 172.21.15.96:0/3220463430 learned_addr learned my addr 172.21.15.96:0/3220463430 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64469e8640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd598c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64461e7640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd597700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 -- 172.21.15.96:0/3220463430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 msgr2=0x55d7fd597700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd597700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 -- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 msgr2=0x55d7fd598c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd598c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 -- 172.21.15.96:0/3220463430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fd599240 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64469e8640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd598c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f64459e6640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70df3f92de139dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f6429ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430021040 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f643000fc20 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.943+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7fd4726a0 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f643002fa30 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 10 client.17022 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d7fd605020 con 0x55d7fd589240 2024-03-18T15:46:36.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d7fd4726a0 con 0x55d7fd589240 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f643002f440 con 0x55d7fd589240 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 10 client.17022 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 10 client.17022 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 20 client.17022 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd605900 0x55d7fd607d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 4 client.17022 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7fd4bedc0 con 0x55d7fd605900 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f64461e7640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd605900 0x55d7fd607d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f6429ffb640 10 client.17022 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.944+0000 7f64461e7640 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd605900 0x55d7fd607d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6ffef1d7d5a93a4 server_cookie=e7ce3cc20ab895fd in_seq=0 out_seq=0 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f6429ffb640 1 -- 172.21.15.96:0/3220463430 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7fd4bedc0 con 0x55d7fd605900 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f6429ffb640 10 client.17022 handle_command_reply: tid=0 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 client.17022 shutdown 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 msgr2=0x55d7fd597c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430009f60 comp rx=0 tx=0).stop 2024-03-18T15:46:36.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 shutdown_connections 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55d7fd605900 0x55d7fd607d20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7fd58e3b0 0x55d7fd598c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f6429ffb640 0 client.17022 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d7fd589240 0x55d7fd597c40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 --2- 172.21.15.96:0/3220463430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7fd588850 0x55d7fd597700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 >> 172.21.15.96:0/3220463430 conn(0x55d7fd583dc0 msgr2=0x55d7fd5878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 shutdown_connections 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 1 -- 172.21.15.96:0/3220463430 wait complete. 2024-03-18T15:46:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644b036740 20 client.17022 trim_cache size 0 max 16384 2024-03-18T15:46:36.953 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:46:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 msgr2=0x7f642c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 0x7f642c0a4cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55d7fd589aa0 tx=0x7f6438009500 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 msgr2=0x7f6440197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f643c000c00 tx=0x7f643c02c650 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 shutdown_connections 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f642c0a2800 0x7f642c0a4cc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64401019a0 0x7f644019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6440100fe0 0x7f6440198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.945+0000 7f644844b640 1 --2- 172.21.15.96:0/1409279557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64400ffe90 0x7f6440197af0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.946+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 >> 172.21.15.96:0/1409279557 conn(0x7f64400fb3e0 msgr2=0x7f6440104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.946+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 shutdown_connections 2024-03-18T15:46:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:36.946+0000 7f644844b640 1 -- 172.21.15.96:0/1409279557 wait complete. 2024-03-18T15:46:36.957 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:46:37.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 -- 172.21.15.96:0/144348407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241006e0 msgr2=0x7faa24100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/144348407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241006e0 0x7faa24100b00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faa18009900 tx=0x7faa1802bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 -- 172.21.15.96:0/144348407 shutdown_connections 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/144348407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 0x7faa24108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/144348407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241018d0 0x7faa24101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/144348407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241006e0 0x7faa24100b00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.061+0000 7faa2a1f1640 1 -- 172.21.15.96:0/144348407 >> 172.21.15.96:0/144348407 conn(0x7faa240fbc70 msgr2=0x7faa240fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- 172.21.15.96:0/144348407 shutdown_connections 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- 172.21.15.96:0/144348407 wait complete. 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 Processor -- start 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- start start 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241006e0 0x7faa240694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 0x7faa2419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410c330 con 0x7faa241018d0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410c1b0 con 0x7faa241006e0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.062+0000 7faa2a1f1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410c4b0 con 0x7faa24102290 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43218/0 (socket says 172.21.15.96:43218) 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 -- 172.21.15.96:0/1769460959 learned_addr learned my addr 172.21.15.96:0/1769460959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa23fff640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 0x7faa2419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa237fe640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241006e0 0x7faa240694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 -- 172.21.15.96:0/1769460959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 msgr2=0x7faa2419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 0x7faa2419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 -- 172.21.15.96:0/1769460959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241006e0 msgr2=0x7faa240694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241006e0 0x7faa240694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 -- 172.21.15.96:0/1769460959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2419f800 con 0x7faa241018d0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa22ffd640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e9f0 tx=0x7faa1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa20ff9640 1 -- 172.21.15.96:0/1769460959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400cd80 con 0x7faa241018d0 2024-03-18T15:46:37.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2419fa90 con 0x7faa241018d0 2024-03-18T15:46:37.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa20ff9640 1 -- 172.21.15.96:0/1769460959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa14010910 con 0x7faa241018d0 2024-03-18T15:46:37.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa20ff9640 1 -- 172.21.15.96:0/1769460959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140188f0 con 0x7faa241018d0 2024-03-18T15:46:37.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.063+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2419ffe0 con 0x7faa241018d0 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.065+0000 7faa20ff9640 1 -- 172.21.15.96:0/1769460959 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7faa14010ac0 con 0x7faa241018d0 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 Processor -- start 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 -- start start 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 0x5612e0cc9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cd1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612e0c2e490 con 0x5612e0cc9850 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5612e0bd6c90 con 0x5612e0cca240 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612e0c2a530 con 0x5612e0ccf3b0 2024-03-18T15:46:37.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa20ff9640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 0x7faa0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa23fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 0x5612e0cc9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cd1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54166/0 (socket says 172.21.15.96:54166) 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 -- 172.21.15.96:0/298488780 learned_addr learned my addr 172.21.15.96:0/298488780 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa23fff640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 0x7faa0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa20ff9640 1 -- 172.21.15.96:0/1769460959 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa140cee40 con 0x7faa241018d0 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 -- 172.21.15.96:0/298488780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 msgr2=0x5612e0cd1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cd1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 -- 172.21.15.96:0/298488780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 msgr2=0x5612e0cc9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 0x5612e0cc9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa237fe640 1 -- 172.21.15.96:0/298488780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e0c91500 con 0x5612e0cca240 2024-03-18T15:46:37.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.066+0000 7faa23fff640 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 0x5612e0cc9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa23fff640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 0x7faa0c0a4cc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009fd0 tx=0x7faa1c004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa237fe640 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa18009900 tx=0x7faa180095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d893dd3444f7996 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa09ffb640 1 -- 172.21.15.96:0/298488780 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1802cb80 con 0x5612e0cca240 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa09ffb640 1 -- 172.21.15.96:0/298488780 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa18005040 con 0x5612e0cca240 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa09ffb640 1 -- 172.21.15.96:0/298488780 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18004920 con 0x5612e0cca240 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 -- 172.21.15.96:0/298488780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 msgr2=0x5612e0cced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faa18009900 tx=0x7faa180095e0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 -- 172.21.15.96:0/298488780 shutdown_connections 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cd1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cced40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/298488780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cc9850 0x5612e0cc9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.067+0000 7faa2cdd8740 1 -- 172.21.15.96:0/298488780 >> 172.21.15.96:0/298488780 conn(0x5612e0cc4dc0 msgr2=0x5612e0cc51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- 172.21.15.96:0/298488780 shutdown_connections 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- 172.21.15.96:0/298488780 wait complete. 2024-03-18T15:46:37.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- start start 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 Processor -- start 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cc9850 0x5612e0cd3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612e0cd2300 con 0x5612e0cca240 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5612e0bea230 con 0x5612e0cc9850 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5612e0cd2480 con 0x5612e0ccf3b0 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa2cdd8740 10 client.0 _ll_register_callbacks cb 0x5612e0c26010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43244/0 (socket says 172.21.15.96:43244) 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa22ffd640 1 -- 172.21.15.96:0/2207392158 learned_addr learned my addr 172.21.15.96:0/2207392158 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa237fe640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cc9850 0x5612e0cd3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.068+0000 7faa23fff640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 -- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 msgr2=0x5612e0cdb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cdb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 -- 172.21.15.96:0/2207392158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cc9850 msgr2=0x5612e0cd3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cc9850 0x5612e0cd3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 -- 172.21.15.96:0/2207392158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e0bf9100 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa22ffd640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa14096300 tx=0x7faa1406a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa0b09cecd3c159 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140a3f50 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa140a3ac0 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa18005040 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa140a3cc0 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa2cdd8740 10 client.17034 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5612e09423f0 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa18005040 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.069+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7faa140d0070 con 0x5612e0cca240 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 10 client.17034 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 10 client.17034 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 20 client.17034 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0cdfad0 0x5612e0ce1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 4 client.17034 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e0c3b9d0 con 0x5612e0cdfad0 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa237fe640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0cdfad0 0x5612e0ce1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa237fe640 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0cdfad0 0x5612e0ce1f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b4e700e880533f2 server_cookie=3919aa9723b7942a in_seq=0 out_seq=0 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.070+0000 7faa02ffd640 10 client.17034 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa02ffd640 1 -- 172.21.15.96:0/2207392158 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7faa180509c0 con 0x5612e0cdfad0 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa02ffd640 10 client.17034 handle_command_reply: tid=0 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 client.17034 shutdown 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 msgr2=0x5612e0cd42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa14096300 tx=0x7faa1406a780 comp rx=0 tx=0).stop 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 shutdown_connections 2024-03-18T15:46:37.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0cdfad0 0x5612e0ce1f40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0ccf3b0 0x5612e0cdb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd42a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa02ffd640 0 client.17034 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2207392158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cc9850 0x5612e0cd3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 >> 172.21.15.96:0/2207392158 conn(0x5612e0cc4dc0 msgr2=0x5612e0cc3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 shutdown_connections 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2207392158 wait complete. 2024-03-18T15:46:37.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.071+0000 7faa2cdd8740 20 client.17034 trim_cache size 0 max 16384 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 Processor -- start 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 -- start start 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cc8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cdb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa241a8e40 con 0x5612e0ccf3b0 2024-03-18T15:46:37.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa241aaa40 con 0x5612e0cca240 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa241a9540 con 0x5612e0cc9850 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43246/0 (socket says 172.21.15.96:43246) 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 -- 172.21.15.96:0/2529532117 learned_addr learned my addr 172.21.15.96:0/2529532117 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa237fe640 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cc8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa22ffd640 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cdb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 -- 172.21.15.96:0/2529532117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 msgr2=0x5612e0cc8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cc8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 -- 172.21.15.96:0/2529532117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 msgr2=0x5612e0cdb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cdb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 -- 172.21.15.96:0/2529532117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e0a0c7d0 con 0x5612e0ccf3b0 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.085+0000 7faa23fff640 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faa1c003b50 tx=0x7faa1c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160273119d978fc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa097fa640 1 -- 172.21.15.96:0/2529532117 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c012cf0 con 0x5612e0ccf3b0 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa097fa640 1 -- 172.21.15.96:0/2529532117 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa1c012e90 con 0x5612e0ccf3b0 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa097fa640 1 -- 172.21.15.96:0/2529532117 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c00fbb0 con 0x5612e0ccf3b0 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2529532117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 msgr2=0x5612e0ce67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faa1c003b50 tx=0x7faa1c00f000 comp rx=0 tx=0).stop 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2529532117 shutdown_connections 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0ccf3b0 0x5612e0ce67d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0cca240 0x5612e0cdb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/2529532117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cc8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2529532117 >> 172.21.15.96:0/2529532117 conn(0x5612e0cc4dc0 msgr2=0x5612e0cc8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2529532117 shutdown_connections 2024-03-18T15:46:37.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.086+0000 7faa2cdd8740 1 -- 172.21.15.96:0/2529532117 wait complete. 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 -- start start 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 Processor -- start 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cd8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0ccf3b0 0x5612e0cda130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa241aa370 con 0x5612e0cca240 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa241a9ca0 con 0x5612e0ccf3b0 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faa241a6e20 con 0x5612e0cc9850 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa2cdd8740 10 client.0 _ll_register_callbacks cb 0x7faa1400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43250/0 (socket says 172.21.15.96:43250) 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 -- 172.21.15.96:0/3333737273 learned_addr learned my addr 172.21.15.96:0/3333737273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa237fe640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cd8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa23fff640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0ccf3b0 0x5612e0cda130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 -- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 msgr2=0x5612e0cd8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cd8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 -- 172.21.15.96:0/3333737273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0ccf3b0 msgr2=0x5612e0cda130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0ccf3b0 0x5612e0cda130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.087+0000 7faa22ffd640 1 -- 172.21.15.96:0/3333737273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e0cda720 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa22ffd640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c6a0 tx=0x7faa14005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f750da885d36d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa140a3f50 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7faa1406d170 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa1c012e90 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faa14097b00 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 10 client.17040 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5612e0d46020 con 0x5612e0cca240 2024-03-18T15:46:37.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa1c012e90 con 0x5612e0cca240 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7faa14097770 con 0x5612e0cca240 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 10 client.17040 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 10 client.17040 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 20 client.17040 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0d46a20 0x5612e0d48e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 4 client.17040 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612e0c35d20 con 0x5612e0d46a20 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.088+0000 7faa237fe640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0d46a20 0x5612e0d48e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa02ffd640 10 client.17040 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa237fe640 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0d46a20 0x5612e0d48e40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9a44b05dbe5210f server_cookie=5f97f53f646dcb82 in_seq=0 out_seq=0 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa02ffd640 1 -- 172.21.15.96:0/3333737273 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612e0c35d20 con 0x5612e0d46a20 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa02ffd640 10 client.17040 handle_command_reply: tid=0 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 client.17040 shutdown 2024-03-18T15:46:37.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 msgr2=0x5612e0cd9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c6a0 tx=0x7faa14005f10 comp rx=0 tx=0).stop 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 shutdown_connections 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5612e0d46a20 0x5612e0d48e40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5612e0ccf3b0 0x5612e0cda130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa02ffd640 0 client.17040 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612e0cca240 0x5612e0cd9120 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 --2- 172.21.15.96:0/3333737273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5612e0cc9850 0x5612e0cd8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 >> 172.21.15.96:0/3333737273 conn(0x5612e0cc4dc0 msgr2=0x5612e0cc88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 shutdown_connections 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 1 -- 172.21.15.96:0/3333737273 wait complete. 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.089+0000 7faa2cdd8740 20 client.17040 trim_cache size 0 max 16384 2024-03-18T15:46:37.100 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:46:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 msgr2=0x7faa0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 0x7faa0c0a4cc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009fd0 tx=0x7faa1c004ab0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 msgr2=0x7faa2419c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faa1400e9f0 tx=0x7faa1400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 shutdown_connections 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7faa0c0a2800 0x7faa0c0a4cc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa24102290 0x7faa2419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa241018d0 0x7faa2419c8d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 --2- 172.21.15.96:0/1769460959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa241006e0 0x7faa240694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 >> 172.21.15.96:0/1769460959 conn(0x7faa240fbc70 msgr2=0x7faa24104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 shutdown_connections 2024-03-18T15:46:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.090+0000 7faa2a1f1640 1 -- 172.21.15.96:0/1769460959 wait complete. 2024-03-18T15:46:37.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:46:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: pgmap v547: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 880 KiB/s wr, 448 op/s 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:46:37.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: pgmap v547: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 880 KiB/s wr, 448 op/s 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:46:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:46:37.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 -- 172.21.15.96:0/124165151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01006e0 msgr2=0x7f3ff0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/124165151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01006e0 0x7f3ff0100b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009900 tx=0x7f3fec02bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 -- 172.21.15.96:0/124165151 shutdown_connections 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/124165151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/124165151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01018d0 0x7f3ff0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/124165151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01006e0 0x7f3ff0100b00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 -- 172.21.15.96:0/124165151 >> 172.21.15.96:0/124165151 conn(0x7f3ff00fbc70 msgr2=0x7f3ff00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 -- 172.21.15.96:0/124165151 shutdown_connections 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.209+0000 7f3ff73d4640 1 -- 172.21.15.96:0/124165151 wait complete. 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 Processor -- start 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 -- start start 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01006e0 0x7f3ff010b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff010d480 con 0x7f3ff01018d0 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff010d300 con 0x7f3ff0102290 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff73d4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff010d600 con 0x7f3ff01006e0 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43274/0 (socket says 172.21.15.96:43274) 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 -- 172.21.15.96:0/367530295 learned_addr learned my addr 172.21.15.96:0/367530295 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff5170640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01006e0 0x7f3ff010b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff5971640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 -- 172.21.15.96:0/367530295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01006e0 msgr2=0x7f3ff010b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01006e0 0x7f3ff010b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 -- 172.21.15.96:0/367530295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 msgr2=0x7f3ff0109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 -- 172.21.15.96:0/367530295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff010a470 con 0x7f3ff01018d0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff5971640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.210+0000 7f3ff496f640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000e9f0 tx=0x7f3fe000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.211+0000 7f3fe67fc640 1 -- 172.21.15.96:0/367530295 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000cd80 con 0x7f3ff01018d0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.211+0000 7f3fe67fc640 1 -- 172.21.15.96:0/367530295 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe0005500 con 0x7f3ff01018d0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.211+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff01a1ca0 con 0x7f3ff01018d0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.211+0000 7f3fe67fc640 1 -- 172.21.15.96:0/367530295 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0010690 con 0x7f3ff01018d0 2024-03-18T15:46:37.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.211+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff01a21a0 con 0x7f3ff01018d0 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3fe67fc640 1 -- 172.21.15.96:0/367530295 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f3fe00046c0 con 0x7f3ff01018d0 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 Processor -- start 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 -- start start 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f04d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f07840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563719e64490 con 0x563719f053b0 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563719e0cc90 con 0x563719eff850 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563719e60530 con 0x563719f00240 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff496f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f07840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff5170640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f04d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54222/0 (socket says 172.21.15.96:54222) 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3ff5971640 1 -- 172.21.15.96:0/3794475242 learned_addr learned my addr 172.21.15.96:0/3794475242 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.213+0000 7f3fe67fc640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 0x7f3fdc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5971640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 0x7f3fdc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 -- 172.21.15.96:0/3794475242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 msgr2=0x563719effc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 -- 172.21.15.96:0/3794475242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 msgr2=0x563719f07840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f07840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 -- 172.21.15.96:0/3794475242 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563719ec7500 con 0x563719f00240 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5971640 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3fe67fc640 1 -- 172.21.15.96:0/367530295 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3fe00cef10 con 0x7f3ff01018d0 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5170640 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f04d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02c170 tx=0x7f3fec02c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65fdff0974ef461c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3ff5971640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 0x7f3fdc0a4cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563719f00aa0 tx=0x7f3fe8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3fdb7fe640 1 -- 172.21.15.96:0/3794475242 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec02cb80 con 0x563719f00240 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3fdb7fe640 1 -- 172.21.15.96:0/3794475242 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fec005040 con 0x563719f00240 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.214+0000 7f3fdb7fe640 1 -- 172.21.15.96:0/3794475242 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0049b0 con 0x563719f00240 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3794475242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 msgr2=0x563719f04d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f04d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02c170 tx=0x7f3fec02c660 comp rx=0 tx=0).stop 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3794475242 shutdown_connections 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f07840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f04d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3794475242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719effc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3794475242 >> 172.21.15.96:0/3794475242 conn(0x563719efadc0 msgr2=0x563719efb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3794475242 shutdown_connections 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3794475242 wait complete. 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 -- start start 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.215+0000 7f3ff9fc0740 1 Processor -- start 2024-03-18T15:46:37.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f09d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563719f08300 con 0x563719f053b0 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563719e20230 con 0x563719eff850 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563719f08480 con 0x563719f00240 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff9fc0740 10 client.0 _ll_register_callbacks cb 0x563719e5c010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff496f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f09d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43294/0 (socket says 172.21.15.96:43294) 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5971640 1 -- 172.21.15.96:0/1653702828 learned_addr learned my addr 172.21.15.96:0/1653702828 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 -- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 msgr2=0x563719f0a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 -- 172.21.15.96:0/1653702828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 msgr2=0x563719f11270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 -- 172.21.15.96:0/1653702828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563719e2f100 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5971640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3ff5170640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f09d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563719efbd30 tx=0x7f3fec02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3c7f640aae6210 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.216+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec04d060 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3fec004730 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fec005040 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3fec005600 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 10 client.26858 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563719b783f0 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fec005040 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3fec003a30 con 0x563719eff850 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 10 client.26858 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 10 client.26858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 20 client.26858 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f15780 0x563719f17b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 4 client.26858 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563719e719d0 con 0x563719f15780 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.217+0000 7f3ff496f640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f15780 0x563719f17b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.218+0000 7f3fd8ff9640 10 client.26858 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.218+0000 7f3ff496f640 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f15780 0x563719f17b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3304a2f41ef2959 server_cookie=74c8b2b8f57a1483 in_seq=0 out_seq=0 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.218+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/1653702828 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563719e719d0 con 0x563719f15780 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.218+0000 7f3fd8ff9640 10 client.26858 handle_command_reply: tid=0 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 client.26858 shutdown 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 msgr2=0x563719f09d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f09d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x563719efbd30 tx=0x7f3fec02bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 shutdown_connections 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f15780 0x563719f17b70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f11270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3fd8ff9640 0 client.26858 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:46:37.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/1653702828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f09d30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 >> 172.21.15.96:0/1653702828 conn(0x563719efadc0 msgr2=0x563719ef9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 shutdown_connections 2024-03-18T15:46:37.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/1653702828 wait complete. 2024-03-18T15:46:37.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.219+0000 7f3ff9fc0740 20 client.26858 trim_cache size 0 max 16384 2024-03-18T15:46:37.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 Processor -- start 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 -- start start 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f04d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719ef9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01aaf10 con 0x563719f053b0 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01acb10 con 0x563719eff850 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.232+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01ab610 con 0x563719f00240 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43310/0 (socket says 172.21.15.96:43310) 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 -- 172.21.15.96:0/3553155525 learned_addr learned my addr 172.21.15.96:0/3553155525 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff496f640 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719ef9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5170640 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f04d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 -- 172.21.15.96:0/3553155525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 msgr2=0x563719ef9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719ef9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 -- 172.21.15.96:0/3553155525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 msgr2=0x563719f04d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f04d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 -- 172.21.15.96:0/3553155525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563719c427d0 con 0x563719f053b0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3ff5971640 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80049a0 tx=0x7f3fe800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f125a5fbf9e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3fdaffd640 1 -- 172.21.15.96:0/3553155525 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8028070 con 0x563719f053b0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3fdaffd640 1 -- 172.21.15.96:0/3553155525 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8003e00 con 0x563719f053b0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.233+0000 7f3fdaffd640 1 -- 172.21.15.96:0/3553155525 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8023630 con 0x563719f053b0 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3553155525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 msgr2=0x563719ef9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80049a0 tx=0x7f3fe800f3c0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3553155525 shutdown_connections 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719ef9c90 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719ef9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/3553155525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f04d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3553155525 >> 172.21.15.96:0/3553155525 conn(0x563719efadc0 msgr2=0x563719efb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3553155525 shutdown_connections 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/3553155525 wait complete. 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 -- start start 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.234+0000 7f3ff9fc0740 1 Processor -- start 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f0e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01ac440 con 0x563719f053b0 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01abd70 con 0x563719eff850 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01a8ef0 con 0x563719f00240 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 10 client.0 _ll_register_callbacks cb 0x7f3fe0097950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43326/0 (socket says 172.21.15.96:43326) 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 -- 172.21.15.96:0/449733706 learned_addr learned my addr 172.21.15.96:0/449733706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5170640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f0e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff496f640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 -- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 msgr2=0x563719f0eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 -- 172.21.15.96:0/449733706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 msgr2=0x563719f0e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f0e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 -- 172.21.15.96:0/449733706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563719f10160 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff5971640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8003fb0 tx=0x7f3fe8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a5bb5c38cae763 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8028070 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f3fe8024440 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe8003e00 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3fe8024640 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 10 client.17058 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.235+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563719f7ba70 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8003e00 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3fe8040430 con 0x563719f053b0 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 10 client.17058 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 10 client.17058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 20 client.17058 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f7c470 0x563719f7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 4 client.17058 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:46:37.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563719e6bd20 con 0x563719f7c470 2024-03-18T15:46:37.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff5170640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f7c470 0x563719f7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3fd8ff9640 10 client.17058 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:46:37.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.236+0000 7f3ff5170640 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f7c470 0x563719f7e890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=daef4b473cb4ca3b server_cookie=427c159db6ef22d2 in_seq=0 out_seq=0 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3fd8ff9640 1 -- 172.21.15.96:0/449733706 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563719e6bd20 con 0x563719f7c470 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3fd8ff9640 10 client.17058 handle_command_reply: tid=0 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 client.17058 shutdown 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 msgr2=0x563719f0fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8003fb0 tx=0x7f3fe8009f90 comp rx=0 tx=0).stop 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 shutdown_connections 2024-03-18T15:46:37.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x563719f7c470 0x563719f7e890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563719f053b0 0x563719f0fb70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3fd8ff9640 0 client.17058 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563719f00240 0x563719f0eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 --2- 172.21.15.96:0/449733706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563719eff850 0x563719f0e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.237+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 >> 172.21.15.96:0/449733706 conn(0x563719efadc0 msgr2=0x563719f09aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 shutdown_connections 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff9fc0740 1 -- 172.21.15.96:0/449733706 wait complete. 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff9fc0740 20 client.17058 trim_cache size 0 max 16384 2024-03-18T15:46:37.245 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 msgr2=0x7f3fdc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 0x7f3fdc0a4cc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x563719f00aa0 tx=0x7f3fe8004ab0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 msgr2=0x7f3ff01096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000e9f0 tx=0x7f3fe000edf0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 shutdown_connections 2024-03-18T15:46:37.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3fdc0a2800 0x7f3fdc0a4cc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ff0102290 0x7f3ff0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ff01018d0 0x7f3ff01096a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 --2- 172.21.15.96:0/367530295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff01006e0 0x7f3ff010b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 >> 172.21.15.96:0/367530295 conn(0x7f3ff00fbc70 msgr2=0x7f3ff0104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 shutdown_connections 2024-03-18T15:46:37.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.238+0000 7f3ff73d4640 1 -- 172.21.15.96:0/367530295 wait complete. 2024-03-18T15:46:37.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:46:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/250150528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec104b10 msgr2=0x7f7bec106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/250150528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec104b10 0x7f7bec106f00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009900 tx=0x7f7be002bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/250150528 shutdown_connections 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/250150528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec107440 0x7f7bec109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/250150528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec104b10 0x7f7bec106f00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/250150528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec1021e0 0x7f7bec1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/250150528 >> 172.21.15.96:0/250150528 conn(0x7f7bec0fbb50 msgr2=0x7f7bec0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/250150528 shutdown_connections 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/250150528 wait complete. 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.355+0000 7f7bf2b9f640 1 Processor -- start 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 -- start start 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 0x7f7bec199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec104b10 0x7f7bec19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec10d330 con 0x7f7bec107440 2024-03-18T15:46:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec10d1b0 con 0x7f7bec1021e0 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf2b9f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec10d4b0 con 0x7f7bec104b10 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43348/0 (socket says 172.21.15.96:43348) 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 -- 172.21.15.96:0/3759480359 learned_addr learned my addr 172.21.15.96:0/3759480359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec104b10 0x7f7bec19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 0x7f7bec199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 -- 172.21.15.96:0/3759480359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec104b10 msgr2=0x7f7bec19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec104b10 0x7f7bec19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 -- 172.21.15.96:0/3759480359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 msgr2=0x7f7bec199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 0x7f7bec199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 -- 172.21.15.96:0/3759480359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bec1a1800 con 0x7f7bec107440 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 0x7f7bec199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.356+0000 7f7bf239e640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e970 tx=0x7f7bdc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.357+0000 7f7beaffd640 1 -- 172.21.15.96:0/3759480359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00ccc0 con 0x7f7bec107440 2024-03-18T15:46:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.357+0000 7f7beaffd640 1 -- 172.21.15.96:0/3759480359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bdc00ce60 con 0x7f7bec107440 2024-03-18T15:46:37.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.357+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bec1a1af0 con 0x7f7bec107440 2024-03-18T15:46:37.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.357+0000 7f7beaffd640 1 -- 172.21.15.96:0/3759480359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc010490 con 0x7f7bec107440 2024-03-18T15:46:37.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.357+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bec1a1fc0 con 0x7f7bec107440 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.359+0000 7f7beaffd640 1 -- 172.21.15.96:0/3759480359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7bdc005020 con 0x7f7bec107440 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.359+0000 7f7bf69ea740 1 Processor -- start 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 -- start start 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334311e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d334319a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d334276490 con 0x55d3343175a0 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d33421ec90 con 0x55d334312430 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d334272530 con 0x55d334311a40 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7beaffd640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 0x7f7bd40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334311e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54272/0 (socket says 172.21.15.96:54272) 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf239e640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 0x7f7bd40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 -- 172.21.15.96:0/3292617151 learned_addr learned my addr 172.21.15.96:0/3292617151 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d334319a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7beaffd640 1 -- 172.21.15.96:0/3759480359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7bdc0cef00 con 0x7f7bec107440 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 -- 172.21.15.96:0/3292617151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 msgr2=0x55d334311e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334311e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 -- 172.21.15.96:0/3292617151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 msgr2=0x55d334319a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d334319a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 -- 172.21.15.96:0/3292617151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3342d9500 con 0x55d334312430 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf239e640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 0x7f7bd40a4cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc09b800 tx=0x7f7bdc06e5a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.360+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7be4011860 tx=0x7f7be4011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c69237403b0b9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bd3fff640 1 -- 172.21.15.96:0/3292617151 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be40112f0 con 0x55d334312430 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bd3fff640 1 -- 172.21.15.96:0/3292617151 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be4003e50 con 0x55d334312430 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bd3fff640 1 -- 172.21.15.96:0/3292617151 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be4004520 con 0x55d334312430 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3292617151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 msgr2=0x55d334316f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7be4011860 tx=0x7f7be4011d20 comp rx=0 tx=0).stop 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3292617151 shutdown_connections 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d334319a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334316f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3292617151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334311e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3292617151 >> 172.21.15.96:0/3292617151 conn(0x55d33430cfb0 msgr2=0x55d33430d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3292617151 shutdown_connections 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.361+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3292617151 wait complete. 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 -- start start 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 Processor -- start 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33431c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d33431c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d33431a550 con 0x55d3343175a0 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d334232230 con 0x55d334312430 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d33431a6d0 con 0x55d334311a40 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf69ea740 10 client.0 _ll_register_callbacks cb 0x55d33426e010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43366/0 (socket says 172.21.15.96:43366) 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf139c640 1 -- 172.21.15.96:0/3293719414 learned_addr learned my addr 172.21.15.96:0/3293719414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf1b9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d33431c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.362+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33431c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 -- 172.21.15.96:0/3293719414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 msgr2=0x55d33431c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33431c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 -- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 msgr2=0x55d33431c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d33431c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 -- 172.21.15.96:0/3293719414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d334241100 con 0x55d3343175a0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d33431c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7bec069400 tx=0x7f7be002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f2125da161dff6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0003c10 con 0x55d3343175a0 2024-03-18T15:46:37.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7be0003db0 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be4003e50 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7be0054030 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 10 client.17070 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d333f8a3f0 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be4003e50 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7be0034440 con 0x55d3343175a0 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 10 client.17070 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 10 client.17070 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.363+0000 7f7bf69ea740 20 client.17070 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d334327e40 0x55d33432a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bf69ea740 4 client.17070 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3342839d0 con 0x55d334327e40 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d334327e40 0x55d33432a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bd17fa640 10 client.17070 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.364+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d334327e40 0x55d33432a2c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78042114ce21542 server_cookie=8e3384f577efe9a in_seq=0 out_seq=0 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bd17fa640 1 -- 172.21.15.96:0/3293719414 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d3342839d0 con 0x55d334327e40 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bd17fa640 10 client.17070 handle_command_reply: tid=0 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 client.17070 shutdown 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 msgr2=0x55d3343235a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7bec069400 tx=0x7f7be002c670 comp rx=0 tx=0).stop 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 shutdown_connections 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343235a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d334327e40 0x55d33432a2c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d33431c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3293719414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33431c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bd17fa640 0 client.17070 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 >> 172.21.15.96:0/3293719414 conn(0x55d33430cfb0 msgr2=0x55d33430c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 shutdown_connections 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3293719414 wait complete. 2024-03-18T15:46:37.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.365+0000 7f7bf69ea740 20 client.17070 trim_cache size 0 max 16384 2024-03-18T15:46:37.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.378+0000 7f7bf69ea740 1 Processor -- start 2024-03-18T15:46:37.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.378+0000 7f7bf69ea740 1 -- start start 2024-03-18T15:46:37.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33430bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334323560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1aaca0 con 0x55d3343175a0 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1ac8a0 con 0x55d334312430 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1ab3a0 con 0x55d334311a40 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43382/0 (socket says 172.21.15.96:43382) 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 -- 172.21.15.96:0/3225250079 learned_addr learned my addr 172.21.15.96:0/3225250079 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33430bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334323560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 -- 172.21.15.96:0/3225250079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 msgr2=0x55d33430bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33430bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 -- 172.21.15.96:0/3225250079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 msgr2=0x55d334323560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334323560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 -- 172.21.15.96:0/3225250079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3340547d0 con 0x55d3343175a0 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bf139c640 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004460 tx=0x7f7be0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af0791c977ee17 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bd37fe640 1 -- 172.21.15.96:0/3225250079 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0034440 con 0x55d3343175a0 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bd37fe640 1 -- 172.21.15.96:0/3225250079 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be005f070 con 0x55d3343175a0 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.379+0000 7f7bd37fe640 1 -- 172.21.15.96:0/3225250079 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0005070 con 0x55d3343175a0 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3225250079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 msgr2=0x55d33432eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004460 tx=0x7f7be0005fd0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3225250079 shutdown_connections 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d33432eb80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d334323560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/3225250079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d33430bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3225250079 >> 172.21.15.96:0/3225250079 conn(0x55d33430cfb0 msgr2=0x55d33430b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3225250079 shutdown_connections 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- 172.21.15.96:0/3225250079 wait complete. 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 -- start start 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.380+0000 7f7bf69ea740 1 Processor -- start 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334320c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d3343211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1ac1d0 con 0x55d3343175a0 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1abb00 con 0x55d334312430 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7bec1a8c80 con 0x55d334311a40 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 10 client.0 _ll_register_callbacks cb 0x7f7be002cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf1b9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d3343211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43394/0 (socket says 172.21.15.96:43394) 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 -- 172.21.15.96:0/1221315699 learned_addr learned my addr 172.21.15.96:0/1221315699 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334320c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 -- 172.21.15.96:0/1221315699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 msgr2=0x55d334320c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334320c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 -- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 msgr2=0x55d3343211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d3343211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 -- 172.21.15.96:0/1221315699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3343227a0 con 0x55d3343175a0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d3343211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf139c640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009900 tx=0x7f7be0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a600fb657cf0534f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0003a10 con 0x55d3343175a0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f7be0034ce0 con 0x55d3343175a0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be005f070 con 0x55d3343175a0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7be0005020 con 0x55d3343175a0 2024-03-18T15:46:37.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.381+0000 7f7bf69ea740 10 client.17076 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d33438e1c0 con 0x55d3343175a0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7be005f070 con 0x55d3343175a0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f7be0004a20 con 0x55d3343175a0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 10 client.17076 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 10 client.17076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 20 client.17076 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d33438ea30 0x55d334390e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 4 client.17076 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d33427dd20 con 0x55d33438ea30 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d33438ea30 0x55d334390e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bd17fa640 10 client.17076 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.382+0000 7f7bf1b9d640 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d33438ea30 0x55d334390e50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6460878be9454007 server_cookie=89fbd075be78bb8b in_seq=0 out_seq=0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bd17fa640 1 -- 172.21.15.96:0/1221315699 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d33427dd20 con 0x55d33438ea30 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bd17fa640 10 client.17076 handle_command_reply: tid=0 2024-03-18T15:46:37.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 client.17076 shutdown 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 msgr2=0x55d3343221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009900 tx=0x7f7be0005e50 comp rx=0 tx=0).stop 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 shutdown_connections 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3343175a0 0x55d3343221b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d33438ea30 0x55d334390e50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d334312430 0x55d3343211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 --2- 172.21.15.96:0/1221315699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d334311a40 0x55d334320c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bd17fa640 0 client.17076 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 >> 172.21.15.96:0/1221315699 conn(0x55d33430cfb0 msgr2=0x55d33430c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 shutdown_connections 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 1 -- 172.21.15.96:0/1221315699 wait complete. 2024-03-18T15:46:37.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf69ea740 20 client.17076 trim_cache size 0 max 16384 2024-03-18T15:46:37.392 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 msgr2=0x7f7bd40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 0x7f7bd40a4cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc09b800 tx=0x7f7bdc06e5a0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 msgr2=0x7f7bec1a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e970 tx=0x7f7bdc00ed70 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 shutdown_connections 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7bd40a2800 0x7f7bd40a4cc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7bec107440 0x7f7bec1a1190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bec104b10 0x7f7bec19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 --2- 172.21.15.96:0/3759480359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bec1021e0 0x7f7bec199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.383+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 >> 172.21.15.96:0/3759480359 conn(0x7f7bec0fbb50 msgr2=0x7f7bec101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.384+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 shutdown_connections 2024-03-18T15:46:37.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.384+0000 7f7bf2b9f640 1 -- 172.21.15.96:0/3759480359 wait complete. 2024-03-18T15:46:37.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 -- 172.21.15.96:0/152274304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee00ffac0 msgr2=0x7efee00ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 --2- 172.21.15.96:0/152274304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee00ffac0 0x7efee00ffee0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efed0009930 tx=0x7efed002bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 -- 172.21.15.96:0/152274304 shutdown_connections 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 --2- 172.21.15.96:0/152274304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee0103450 0x7efee0109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 --2- 172.21.15.96:0/152274304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0102a90 0x7efee0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 --2- 172.21.15.96:0/152274304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee00ffac0 0x7efee00ffee0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 -- 172.21.15.96:0/152274304 >> 172.21.15.96:0/152274304 conn(0x7efee00fb410 msgr2=0x7efee00fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 -- 172.21.15.96:0/152274304 shutdown_connections 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.501+0000 7efee5a4d640 1 -- 172.21.15.96:0/152274304 wait complete. 2024-03-18T15:46:37.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 Processor -- start 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 -- start start 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee00ffac0 0x7efee0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0102a90 0x7efee01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d490 con 0x7efee0102a90 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d310 con 0x7efee00ffac0 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efee5a4d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efee010d610 con 0x7efee0103450 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0102a90 0x7efee01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33274/0 (socket says 172.21.15.96:33274) 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 -- 172.21.15.96:0/1579466800 learned_addr learned my addr 172.21.15.96:0/1579466800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedeffd640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee00ffac0 0x7efee0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 -- 172.21.15.96:0/1579466800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee00ffac0 msgr2=0x7efee0197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee00ffac0 0x7efee0197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 -- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0102a90 msgr2=0x7efee01980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0102a90 0x7efee01980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.502+0000 7efedf7fe640 1 -- 172.21.15.96:0/1579466800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019f7d0 con 0x7efee0103450 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efedf7fe640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efed800eb10 tx=0x7efed800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efee4a4b640 1 -- 172.21.15.96:0/1579466800 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8004b20 con 0x7efee0103450 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019fac0 con 0x7efee0103450 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efee4a4b640 1 -- 172.21.15.96:0/1579466800 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed8004cc0 con 0x7efee0103450 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a0010 con 0x7efee0103450 2024-03-18T15:46:37.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.503+0000 7efee4a4b640 1 -- 172.21.15.96:0/1579466800 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8010760 con 0x7efee0103450 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.505+0000 7efee8631740 1 Processor -- start 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee4a4b640 1 -- 172.21.15.96:0/1579466800 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efed8005020 con 0x7efee0103450 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 -- start start 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558de2649490 con 0x558de26e4850 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558de25f1c90 con 0x558de26ea3b0 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee8631740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558de2645530 con 0x558de26e5240 2024-03-18T15:46:37.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43428/0 (socket says 172.21.15.96:43428) 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 -- 172.21.15.96:0/1361722970 learned_addr learned my addr 172.21.15.96:0/1361722970 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efedeffd640 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee4a4b640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 0x7efec80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 -- 172.21.15.96:0/1361722970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 msgr2=0x558de26e9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26e9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efedf7fe640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 0x7efec80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 -- 172.21.15.96:0/1361722970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 msgr2=0x558de26ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26ec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 -- 172.21.15.96:0/1361722970 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de26ac500 con 0x558de26e4850 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efee4a4b640 1 -- 172.21.15.96:0/1579466800 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efed80cf000 con 0x7efee0103450 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.506+0000 7efede7fc640 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efed4013820 tx=0x7efed4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c010319dce154586 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efec5ffb640 1 -- 172.21.15.96:0/1361722970 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4013da0 con 0x558de26e4850 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efec5ffb640 1 -- 172.21.15.96:0/1361722970 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4013120 con 0x558de26e4850 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efec5ffb640 1 -- 172.21.15.96:0/1361722970 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4010a00 con 0x558de26e4850 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 -- 172.21.15.96:0/1361722970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 msgr2=0x558de26e4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efedf7fe640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 0x7efec80a4cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efed80983e0 tx=0x7efed8066720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efed4013820 tx=0x7efed4009780 comp rx=0 tx=0).stop 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 -- 172.21.15.96:0/1361722970 shutdown_connections 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26e9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 --2- 172.21.15.96:0/1361722970 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e4850 0x558de26e4c70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 -- 172.21.15.96:0/1361722970 >> 172.21.15.96:0/1361722970 conn(0x558de26dfdc0 msgr2=0x558de26e01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 -- 172.21.15.96:0/1361722970 shutdown_connections 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.507+0000 7efee8631740 1 -- 172.21.15.96:0/1361722970 wait complete. 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 -- start start 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 Processor -- start 2024-03-18T15:46:37.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26eed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 0x558de26ef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558de26ed300 con 0x558de26ea3b0 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558de2605230 con 0x558de26e5240 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558de26ed480 con 0x558de26e4850 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efee8631740 10 client.0 _ll_register_callbacks cb 0x558de2641010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43442/0 (socket says 172.21.15.96:43442) 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efedeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 0x558de26ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efede7fc640 1 -- 172.21.15.96:0/1951532330 learned_addr learned my addr 172.21.15.96:0/1951532330 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.508+0000 7efede7fc640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 -- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 msgr2=0x558de26eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 -- 172.21.15.96:0/1951532330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 msgr2=0x558de26f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 -- 172.21.15.96:0/1951532330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de2614100 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efede7fc640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efedeffd640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 0x558de26ef290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efed0036040 tx=0x7efed0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa61799b2180419 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0005d50 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efed0005230 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4013120 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efed0005430 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efee8631740 10 client.26894 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558de235d3f0 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.509+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4013120 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efed0003680 con 0x558de26e5240 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 10 client.26894 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 20 client.26894 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de26fab10 0x558de26fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 4 client.26894 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558de26569d0 con 0x558de26fab10 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efede7fc640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de26fab10 0x558de26fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efede7fc640 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de26fab10 0x558de26fcf60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f350f3a0c226c9c server_cookie=80b9b3a43c9d28fd in_seq=0 out_seq=0 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.510+0000 7efebeffd640 10 client.26894 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efebeffd640 1 -- 172.21.15.96:0/1951532330 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x558de26569d0 con 0x558de26fab10 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efebeffd640 10 client.26894 handle_command_reply: tid=0 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 client.26894 shutdown 2024-03-18T15:46:37.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 msgr2=0x558de26ef290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 0x558de26ef290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efed0036040 tx=0x7efed0004270 comp rx=0 tx=0).stop 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 shutdown_connections 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de26fab10 0x558de26fcf60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de26f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e5240 0x558de26ef290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 --2- 172.21.15.96:0/1951532330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efebeffd640 0 client.26894 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 >> 172.21.15.96:0/1951532330 conn(0x558de26dfdc0 msgr2=0x558de26defe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.511+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 shutdown_connections 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.512+0000 7efee8631740 1 -- 172.21.15.96:0/1951532330 wait complete. 2024-03-18T15:46:37.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.512+0000 7efee8631740 20 client.26894 trim_cache size 0 max 16384 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 Processor -- start 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 -- start start 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a8e20 con 0x558de26e5240 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aaa20 con 0x558de26ea3b0 2024-03-18T15:46:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efee8631740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a9520 con 0x558de26e4850 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43446/0 (socket says 172.21.15.96:43446) 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 -- 172.21.15.96:0/3113941435 learned_addr learned my addr 172.21.15.96:0/3113941435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efede7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efede7fc640 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 -- 172.21.15.96:0/3113941435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 msgr2=0x558de26e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26e01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 -- 172.21.15.96:0/3113941435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 msgr2=0x558de26e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 -- 172.21.15.96:0/3113941435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558de24277d0 con 0x558de26e5240 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.525+0000 7efedeffd640 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efed0006010 tx=0x7efed003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35e79bcb19e71c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efec57fa640 1 -- 172.21.15.96:0/3113941435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0005d50 con 0x558de26e5240 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efec57fa640 1 -- 172.21.15.96:0/3113941435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00035f0 con 0x558de26e5240 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efec57fa640 1 -- 172.21.15.96:0/3113941435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0034950 con 0x558de26e5240 2024-03-18T15:46:37.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 -- 172.21.15.96:0/3113941435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 msgr2=0x558de26f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efed0006010 tx=0x7efed003f040 comp rx=0 tx=0).stop 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 -- 172.21.15.96:0/3113941435 shutdown_connections 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26ea3b0 0x558de26e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26e5240 0x558de26f6250 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 --2- 172.21.15.96:0/3113941435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e4850 0x558de26e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 -- 172.21.15.96:0/3113941435 >> 172.21.15.96:0/3113941435 conn(0x558de26dfdc0 msgr2=0x558de26e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 -- 172.21.15.96:0/3113941435 shutdown_connections 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.526+0000 7efee8631740 1 -- 172.21.15.96:0/3113941435 wait complete. 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 -- start start 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 Processor -- start 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e4850 0x558de26f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26f3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aa350 con 0x558de26ea3b0 2024-03-18T15:46:37.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a9c80 con 0x558de26e4850 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a6e00 con 0x558de26e5240 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efee8631740 10 client.0 _ll_register_callbacks cb 0x7efed4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efedeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efedeffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33316/0 (socket says 172.21.15.96:33316) 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43460/0 (socket says 172.21.15.96:43460) 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efedeffd640 1 -- 172.21.15.96:0/1623513066 learned_addr learned my addr 172.21.15.96:0/1623513066 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e4850 0x558de26f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 -- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 msgr2=0x558de26f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 -- 172.21.15.96:0/1623513066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e4850 msgr2=0x558de26f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e4850 0x558de26f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.527+0000 7efede7fc640 1 -- 172.21.15.96:0/1623513066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558de26f6460 con 0x558de26ea3b0 2024-03-18T15:46:37.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efede7fc640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x558de26fd7a0 tx=0x7efed002cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca576262b49e3803 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed402eb30 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7efed4010a90 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00035f0 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7efed40050e0 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 10 client.17094 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558de2760360 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00035f0 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7efed400f820 con 0x558de26ea3b0 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 10 client.17094 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 20 client.17094 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de2760ba0 0x558de2762fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 4 client.17094 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:46:37.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558de2650d20 con 0x558de2760ba0 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.528+0000 7efedeffd640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de2760ba0 0x558de2762fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efebeffd640 10 client.17094 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efedeffd640 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de2760ba0 0x558de2762fc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff1355bfa07be148 server_cookie=35e1ea322e28939 in_seq=0 out_seq=0 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efebeffd640 1 -- 172.21.15.96:0/1623513066 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558de2650d20 con 0x558de2760ba0 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efebeffd640 10 client.17094 handle_command_reply: tid=0 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 client.17094 shutdown 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 msgr2=0x558de275ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x558de26fd7a0 tx=0x7efed002cd80 comp rx=0 tx=0).stop 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 shutdown_connections 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x558de2760ba0 0x558de2762fc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558de26ea3b0 0x558de275ee50 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558de26e5240 0x558de26f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efebeffd640 0 client.17094 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 --2- 172.21.15.96:0/1623513066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558de26e4850 0x558de26f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 >> 172.21.15.96:0/1623513066 conn(0x558de26dfdc0 msgr2=0x558de26ed820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 shutdown_connections 2024-03-18T15:46:37.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 1 -- 172.21.15.96:0/1623513066 wait complete. 2024-03-18T15:46:37.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.529+0000 7efee8631740 20 client.17094 trim_cache size 0 max 16384 2024-03-18T15:46:37.537 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 msgr2=0x7efec80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 0x7efec80a4cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7efed80983e0 tx=0x7efed8066720 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 msgr2=0x7efee019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efed800eb10 tx=0x7efed800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 shutdown_connections 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efec80a2800 0x7efec80a4cc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efee0103450 0x7efee019f160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0102a90 0x7efee01980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 --2- 172.21.15.96:0/1579466800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efee00ffac0 0x7efee0197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:37.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 >> 172.21.15.96:0/1579466800 conn(0x7efee00fb410 msgr2=0x7efee0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:37.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 shutdown_connections 2024-03-18T15:46:37.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:37.530+0000 7efee5a4d640 1 -- 172.21.15.96:0/1579466800 wait complete. 2024-03-18T15:46:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:37 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-03-18T15:46:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:37 smithi096 ceph-mon[31930]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-03-18T15:46:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:37 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4048772731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:46:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:37 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-03-18T15:46:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:37 smithi137 ceph-mon[35868]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-03-18T15:46:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:37 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4048772731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:46:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:37 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-03-18T15:46:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:37 smithi193 ceph-mon[36702]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-03-18T15:46:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:37 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4048772731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:46:38.545 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:46:38.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:46:38.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:46:38.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 -- 172.21.15.96:0/648895036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a00ff860 msgr2=0x7f27a00ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 --2- 172.21.15.96:0/648895036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a00ff860 0x7f27a00ffce0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2794009900 tx=0x7f279402bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 -- 172.21.15.96:0/648895036 shutdown_connections 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 --2- 172.21.15.96:0/648895036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a0100220 0x7f27a0109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 --2- 172.21.15.96:0/648895036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a00ff860 0x7f27a00ffce0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 --2- 172.21.15.96:0/648895036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0102df0 0x7f27a0103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 -- 172.21.15.96:0/648895036 >> 172.21.15.96:0/648895036 conn(0x7f27a00fb3e0 msgr2=0x7f27a00fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 -- 172.21.15.96:0/648895036 shutdown_connections 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.653+0000 7f27a5da9640 1 -- 172.21.15.96:0/648895036 wait complete. 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 Processor -- start 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 -- start start 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a00ff860 0x7f27a0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 0x7f27a0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010d480 con 0x7f27a0102df0 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010d300 con 0x7f27a0100220 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f27a5da9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27a010d600 con 0x7f27a00ff860 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a00ff860 0x7f27a0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43490/0 (socket says 172.21.15.96:43490) 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 -- 172.21.15.96:0/2033859839 learned_addr learned my addr 172.21.15.96:0/2033859839 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279effd640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 0x7f27a0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 -- 172.21.15.96:0/2033859839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a00ff860 msgr2=0x7f27a0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a00ff860 0x7f27a0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 -- 172.21.15.96:0/2033859839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 msgr2=0x7f27a0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 0x7f27a0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279ffff640 1 -- 172.21.15.96:0/2033859839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a019f750 con 0x7f27a0102df0 2024-03-18T15:46:38.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.654+0000 7f279effd640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 0x7f27a0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f279ffff640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f279800eb10 tx=0x7f279800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f279cff9640 1 -- 172.21.15.96:0/2033859839 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798004b20 con 0x7f27a0102df0 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a019fa40 con 0x7f27a0102df0 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f279cff9640 1 -- 172.21.15.96:0/2033859839 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2798004cc0 con 0x7f27a0102df0 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f279cff9640 1 -- 172.21.15.96:0/2033859839 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798010760 con 0x7f27a0102df0 2024-03-18T15:46:38.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.655+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a019ff60 con 0x7f27a0102df0 2024-03-18T15:46:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279cff9640 1 -- 172.21.15.96:0/2033859839 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2798005020 con 0x7f27a0102df0 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 Processor -- start 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 -- start start 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bc34f0 0x5571b4bc5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571b4b220d0 con 0x5571b4bc34f0 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5571b4acac70 con 0x5571b4bbe380 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f27a898e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571b4b1e170 con 0x5571b4bbd990 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bc34f0 0x5571b4bc5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54374/0 (socket says 172.21.15.96:54374) 2024-03-18T15:46:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 -- 172.21.15.96:0/3242103068 learned_addr learned my addr 172.21.15.96:0/3242103068 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:33340/0 (socket says 172.21.15.96:33340) 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279cff9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 0x7f27880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279ffff640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 0x7f27880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 -- 172.21.15.96:0/3242103068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 msgr2=0x5571b4bbddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 -- 172.21.15.96:0/3242103068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bc34f0 msgr2=0x5571b4bc5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bc34f0 0x5571b4bc5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279f7fe640 1 -- 172.21.15.96:0/3242103068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b48fe7e0 con 0x5571b4bbe380 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.658+0000 7f279cff9640 1 -- 172.21.15.96:0/2033859839 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f27980cf000 con 0x7f27a0102df0 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2790009ca0 tx=0x7f2790011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee10b8dbdf71c9ed server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f279ffff640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 0x7f27880a4cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f279809b670 tx=0x7f279806a7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f2785ffb640 1 -- 172.21.15.96:0/3242103068 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790009e50 con 0x5571b4bbe380 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f2785ffb640 1 -- 172.21.15.96:0/3242103068 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f2785ffb640 1 -- 172.21.15.96:0/3242103068 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790019650 con 0x5571b4bbe380 2024-03-18T15:46:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f27a898e740 1 -- 172.21.15.96:0/3242103068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 msgr2=0x5571b4bc2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.659+0000 7f27a898e740 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2790009ca0 tx=0x7f2790011810 comp rx=0 tx=0).stop 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- 172.21.15.96:0/3242103068 shutdown_connections 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bc34f0 0x5571b4bc5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc2e80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- 172.21.15.96:0/3242103068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- 172.21.15.96:0/3242103068 >> 172.21.15.96:0/3242103068 conn(0x5571b4b85c50 msgr2=0x5571b4bb9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- 172.21.15.96:0/3242103068 shutdown_connections 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- 172.21.15.96:0/3242103068 wait complete. 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- start start 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 Processor -- start 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 0x5571b4bc7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bc34f0 0x5571b4bcf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5571b4bc64a0 con 0x5571b4bbd990 2024-03-18T15:46:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5571b4ade0d0 con 0x5571b4bbe380 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5571b4bc6620 con 0x5571b4bc34f0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f27a898e740 10 client.0 _ll_register_callbacks cb 0x5571b4aa7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f279effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 0x5571b4bc7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54390/0 (socket says 172.21.15.96:54390) 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f279effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 0x5571b4bc7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43498/0 (socket says 172.21.15.96:43498) 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.660+0000 7f279f7fe640 1 -- 172.21.15.96:0/3762420972 learned_addr learned my addr 172.21.15.96:0/3762420972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279effd640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bc34f0 0x5571b4bcf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 -- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bc34f0 msgr2=0x5571b4bcf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bc34f0 0x5571b4bcf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 -- 172.21.15.96:0/3762420972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 msgr2=0x5571b4bc7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 0x5571b4bc7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 -- 172.21.15.96:0/3762420972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b495ef70 con 0x5571b4bbe380 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2790009730 tx=0x7f2790004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade9df4972dfe754 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f277effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790028020 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2790016040 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2790033050 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f27a898e740 10 client.26912 fetch_fsmap learned FSMap version 35 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5571b47f6170 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.661+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f2790004990 con 0x5571b4bbe380 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 10 client.26912 fetch_fsmap finished waiting for FSMap version 35 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 10 client.26912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 20 client.26912 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4bd3750 0x5571b4bd5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 4 client.26912 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:38.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b4b2f610 con 0x5571b4bd3750 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f279effd640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4bd3750 0x5571b4bd5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f277effd640 10 client.26912 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.662+0000 7f279effd640 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4bd3750 0x5571b4bd5ba0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21e7bfd5645f7cc server_cookie=50c9beab96e4398a in_seq=0 out_seq=0 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.663+0000 7f277effd640 1 -- 172.21.15.96:0/3762420972 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5571b4b2f610 con 0x5571b4bd3750 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.663+0000 7f277effd640 10 client.26912 handle_command_reply: tid=0 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.663+0000 7f27a898e740 1 client.26912 shutdown 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 msgr2=0x5571b4bc8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2790009730 tx=0x7f2790004230 comp rx=0 tx=0).stop 2024-03-18T15:46:38.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 shutdown_connections 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bc34f0 0x5571b4bcf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bbe380 0x5571b4bc8390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4bd3750 0x5571b4bd5ba0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 --2- 172.21.15.96:0/3762420972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbd990 0x5571b4bc7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f277effd640 0 client.26912 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 >> 172.21.15.96:0/3762420972 conn(0x5571b4b85c50 msgr2=0x5571b4bb84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 shutdown_connections 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 1 -- 172.21.15.96:0/3762420972 wait complete. 2024-03-18T15:46:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.664+0000 7f27a898e740 20 client.26912 trim_cache size 0 max 16384 2024-03-18T15:46:38.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 Processor -- start 2024-03-18T15:46:38.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 -- start start 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bbca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01a8cf0 con 0x5571b4bbe380 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aa8f0 con 0x5571b4bc34f0 2024-03-18T15:46:38.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f27a898e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01a93f0 con 0x5571b4bbd990 2024-03-18T15:46:38.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43512/0 (socket says 172.21.15.96:43512) 2024-03-18T15:46:38.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f279f7fe640 1 -- 172.21.15.96:0/3108118067 learned_addr learned my addr 172.21.15.96:0/3108118067 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f279effd640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bbca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.685+0000 7f279effd640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 -- 172.21.15.96:0/3108118067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 msgr2=0x5571b4bbc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 -- 172.21.15.96:0/3108118067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 msgr2=0x5571b4bbca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bbca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 -- 172.21.15.96:0/3108118067 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b4bc6380 con 0x5571b4bbe380 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279effd640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bbca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:38.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f279f7fe640 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2790011850 tx=0x7f2790005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c3ec96ceb721be server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f27857fa640 1 -- 172.21.15.96:0/3108118067 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790028040 con 0x5571b4bbe380 2024-03-18T15:46:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f27857fa640 1 -- 172.21.15.96:0/3108118067 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.686+0000 7f27857fa640 1 -- 172.21.15.96:0/3108118067 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790005460 con 0x5571b4bbe380 2024-03-18T15:46:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- 172.21.15.96:0/3108118067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 msgr2=0x5571b4bcf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2790011850 tx=0x7f2790005990 comp rx=0 tx=0).stop 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- 172.21.15.96:0/3108118067 shutdown_connections 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bbca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcf390 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 --2- 172.21.15.96:0/3108118067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bbc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- 172.21.15.96:0/3108118067 >> 172.21.15.96:0/3108118067 conn(0x5571b4b85c50 msgr2=0x5571b4bbf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- 172.21.15.96:0/3108118067 shutdown_connections 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- 172.21.15.96:0/3108118067 wait complete. 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 -- start start 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.687+0000 7f27a898e740 1 Processor -- start 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aa220 con 0x5571b4bbe380 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01a9b50 con 0x5571b4bc34f0 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01a6cd0 con 0x5571b4bbd990 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 10 client.0 _ll_register_callbacks cb 0x7f2794009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43528/0 (socket says 172.21.15.96:43528) 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 -- 172.21.15.96:0/2938777710 learned_addr learned my addr 172.21.15.96:0/2938777710 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279effd640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279effd640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 -- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 msgr2=0x5571b4bccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 -- 172.21.15.96:0/2938777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 msgr2=0x5571b4bce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 -- 172.21.15.96:0/2938777710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b4bcea60 con 0x5571b4bbe380 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f279f7fe640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2790009300 tx=0x7f27900097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddcb262f1179d45 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f277effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790028040 con 0x5571b4bbe380 2024-03-18T15:46:38.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f2790004830 con 0x5571b4bbe380 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.688+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2790003e40 con 0x5571b4bbe380 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 10 client.17112 fetch_fsmap learned FSMap version 36 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5571b4c39f10 con 0x5571b4bbe380 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27900035f0 con 0x5571b4bbe380 2024-03-18T15:46:38.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f279004a070 con 0x5571b4bbe380 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 10 client.17112 fetch_fsmap finished waiting for FSMap version 36 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 10 client.17112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 20 client.17112 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4c3a7d0 0x5571b4c3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 4 client.17112 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:46:38.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571b4bb95f0 con 0x5571b4c3a7d0 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f279effd640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4c3a7d0 0x5571b4c3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f279effd640 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4c3a7d0 0x5571b4c3cbf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cf260e365b2a4a server_cookie=2230ef99ecc1753d in_seq=0 out_seq=0 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.689+0000 7f277effd640 10 client.17112 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f277effd640 1 -- 172.21.15.96:0/2938777710 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571b4bb95f0 con 0x5571b4c3a7d0 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f277effd640 10 client.17112 handle_command_reply: tid=0 2024-03-18T15:46:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 client.17112 shutdown 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 msgr2=0x5571b4bcd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2790009300 tx=0x7f27900097a0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 shutdown_connections 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5571b4bc34f0 0x5571b4bce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5571b4c3a7d0 0x5571b4c3cbf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5571b4bbe380 0x5571b4bcd4b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f277effd640 0 client.17112 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 --2- 172.21.15.96:0/2938777710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5571b4bbd990 0x5571b4bccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 >> 172.21.15.96:0/2938777710 conn(0x5571b4b85c50 msgr2=0x5571b4bb7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 shutdown_connections 2024-03-18T15:46:38.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 1 -- 172.21.15.96:0/2938777710 wait complete. 2024-03-18T15:46:38.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.691+0000 7f27a898e740 20 client.17112 trim_cache size 0 max 16384 2024-03-18T15:46:38.713 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:46:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:46:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "8e2b9d5c-e623-4a24-ae76-a771d573bdf2", 2024-03-18T15:46:38.713 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:46:38.714 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 msgr2=0x7f27880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 0x7f27880a4cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f279809b670 tx=0x7f279806a7d0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 msgr2=0x7f27a019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f279800eb10 tx=0x7f279800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 shutdown_connections 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f27880a2800 0x7f27880a4cc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0102df0 0x7f27a019f0e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27a0100220 0x7f27a0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 --2- 172.21.15.96:0/2033859839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27a00ff860 0x7f27a0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 >> 172.21.15.96:0/2033859839 conn(0x7f27a00fb3e0 msgr2=0x7f27a0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 shutdown_connections 2024-03-18T15:46:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.692+0000 7f27a5da9640 1 -- 172.21.15.96:0/2033859839 wait complete. 2024-03-18T15:46:38.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:46:38.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 -- 172.21.15.96:0/930241497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1018d0 msgr2=0x7f413c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 --2- 172.21.15.96:0/930241497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1018d0 0x7f413c101d50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4130009900 tx=0x7f413002bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 -- 172.21.15.96:0/930241497 shutdown_connections 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 --2- 172.21.15.96:0/930241497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 0x7f413c108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 --2- 172.21.15.96:0/930241497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1018d0 0x7f413c101d50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 --2- 172.21.15.96:0/930241497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1006e0 0x7f413c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 -- 172.21.15.96:0/930241497 >> 172.21.15.96:0/930241497 conn(0x7f413c0fbc70 msgr2=0x7f413c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 -- 172.21.15.96:0/930241497 shutdown_connections 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.827+0000 7f4140e97640 1 -- 172.21.15.96:0/930241497 wait complete. 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 Processor -- start 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 -- start start 2024-03-18T15:46:38.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1018d0 0x7f413c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 0x7f413c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10c360 con 0x7f413c1006e0 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10c1e0 con 0x7f413c1018d0 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4140e97640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10c4e0 con 0x7f413c102290 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43552/0 (socket says 172.21.15.96:43552) 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 -- 172.21.15.96:0/614711352 learned_addr learned my addr 172.21.15.96:0/614711352 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f4139d9b640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1018d0 0x7f413c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413ad9d640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 0x7f413c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 -- 172.21.15.96:0/614711352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 msgr2=0x7f413c19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 0x7f413c19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 -- 172.21.15.96:0/614711352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1018d0 msgr2=0x7f413c196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1018d0 0x7f413c196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 -- 172.21.15.96:0/614711352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c19e620 con 0x7f413c1006e0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f413a59c640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e9c0 tx=0x7f412c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.828+0000 7f412b7fe640 1 -- 172.21.15.96:0/614711352 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c00cd80 con 0x7f413c1006e0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.829+0000 7f412b7fe640 1 -- 172.21.15.96:0/614711352 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c0035f0 con 0x7f413c1006e0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.829+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c19e910 con 0x7f413c1006e0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.829+0000 7f412b7fe640 1 -- 172.21.15.96:0/614711352 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c010690 con 0x7f413c1006e0 2024-03-18T15:46:38.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.829+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c19ee30 con 0x7f413c1006e0 2024-03-18T15:46:38.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.831+0000 7f412b7fe640 1 -- 172.21.15.96:0/614711352 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f412c0037a0 con 0x7f413c1006e0 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 Processor -- start 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 -- start start 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7121660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b7124160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2b7081500 con 0x55a2b711c170 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2b7029be0 con 0x55a2b711cb60 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4143a83740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2b707d5a0 con 0x55a2b7121cd0 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f412b7fe640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 0x7f41240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43560/0 (socket says 172.21.15.96:43560) 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413ad9d640 1 -- 172.21.15.96:0/224749154 learned_addr learned my addr 172.21.15.96:0/224749154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413a59c640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7121660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f4139d9b640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b7124160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f412b7fe640 1 -- 172.21.15.96:0/614711352 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f412c0cee40 con 0x7f413c1006e0 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413ad9d640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 0x7f41240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413a59c640 1 -- 172.21.15.96:0/224749154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 msgr2=0x55a2b7124160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413a59c640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b7124160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413a59c640 1 -- 172.21.15.96:0/224749154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 msgr2=0x55a2b711c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.832+0000 7f413a59c640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f413a59c640 1 -- 172.21.15.96:0/224749154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b703d4b0 con 0x55a2b711cb60 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f413ad9d640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f413a59c640 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7121660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f412c0ccc80 tx=0x7f412c06e4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfc5db4eee767ca server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f413ad9d640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 0x7f41240a4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55a2b711d3c0 tx=0x7f4134004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f4128ff9640 1 -- 172.21.15.96:0/224749154 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0960c0 con 0x55a2b711cb60 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f4128ff9640 1 -- 172.21.15.96:0/224749154 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c09beb0 con 0x55a2b711cb60 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.833+0000 7f4128ff9640 1 -- 172.21.15.96:0/224749154 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0860d0 con 0x55a2b711cb60 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- 172.21.15.96:0/224749154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 msgr2=0x55a2b7121660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7121660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f412c0ccc80 tx=0x7f412c06e4e0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- 172.21.15.96:0/224749154 shutdown_connections 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b7124160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7121660 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- 172.21.15.96:0/224749154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- 172.21.15.96:0/224749154 >> 172.21.15.96:0/224749154 conn(0x55a2b71176e0 msgr2=0x55a2b7117ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- 172.21.15.96:0/224749154 shutdown_connections 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- 172.21.15.96:0/224749154 wait complete. 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- start start 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 Processor -- start 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7126b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b712dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2b7124c20 con 0x55a2b711c170 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a2b703d160 con 0x55a2b711cb60 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a2b7124da0 con 0x55a2b7121cd0 2024-03-18T15:46:38.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f4143a83740 10 client.0 _ll_register_callbacks cb 0x55a2b7021150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.834+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43570/0 (socket says 172.21.15.96:43570) 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 -- 172.21.15.96:0/1309047185 learned_addr learned my addr 172.21.15.96:0/1309047185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413ad9d640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b712dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4139d9b640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7126b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 -- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 msgr2=0x55a2b712dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b712dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 -- 172.21.15.96:0/1309047185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 msgr2=0x55a2b7126b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7126b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 -- 172.21.15.96:0/1309047185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b6e5f7d0 con 0x55a2b711c170 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f413a59c640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c6a0 tx=0x7f412c09f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b6389c4fa6d8e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0b4060 con 0x55a2b711c170 2024-03-18T15:46:38.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f412c0860d0 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412c09beb0 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f412c09b7b0 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4143a83740 10 client.17124 fetch_fsmap learned FSMap version 36 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a2b6d953f0 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c09beb0 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.835+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f412c0afb90 con 0x55a2b711c170 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 10 client.17124 fetch_fsmap finished waiting for FSMap version 36 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 20 client.17124 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b7131d30 0x55a2b7134120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 4 client.17124 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2b708ea40 con 0x55a2b7131d30 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4139d9b640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b7131d30 0x55a2b7134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4139d9b640 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b7131d30 0x55a2b7134120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ee47a4478aeca8 server_cookie=90eae4fb171501bf in_seq=0 out_seq=0 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.836+0000 7f4121ffb640 10 client.17124 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4121ffb640 1 -- 172.21.15.96:0/1309047185 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a2b708ea40 con 0x55a2b7131d30 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4121ffb640 10 client.17124 handle_command_reply: tid=0 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 client.17124 shutdown 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 msgr2=0x55a2b7126650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c6a0 tx=0x7f412c09f970 comp rx=0 tx=0).stop 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 shutdown_connections 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b712dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7126b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b7131d30 0x55a2b7134120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 --2- 172.21.15.96:0/1309047185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b7126650 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4121ffb640 0 client.17124 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 >> 172.21.15.96:0/1309047185 conn(0x55a2b71176e0 msgr2=0x55a2b7116900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 shutdown_connections 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 1 -- 172.21.15.96:0/1309047185 wait complete. 2024-03-18T15:46:38.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.837+0000 7f4143a83740 20 client.17124 trim_cache size 0 max 16384 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 Processor -- start 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 -- start start 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b71165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a7d00 con 0x55a2b711c170 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a9930 con 0x55a2b711cb60 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4143a83740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a8400 con 0x55a2b7121cd0 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4139d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4139d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:54464/0 (socket says 172.21.15.96:54464) 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f4139d9b640 1 -- 172.21.15.96:0/2854204047 learned_addr learned my addr 172.21.15.96:0/2854204047 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f413a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43582/0 (socket says 172.21.15.96:43582) 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.855+0000 7f413ad9d640 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b71165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 -- 172.21.15.96:0/2854204047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 msgr2=0x55a2b71165b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b71165b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 -- 172.21.15.96:0/2854204047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 msgr2=0x55a2b711b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 -- 172.21.15.96:0/2854204047 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b6d550f0 con 0x55a2b711cb60 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f413a59c640 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4139d9b640 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f413002cfd0 tx=0x7f4130005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b70aed85810b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4123fff640 1 -- 172.21.15.96:0/2854204047 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413002cd20 con 0x55a2b711cb60 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4123fff640 1 -- 172.21.15.96:0/2854204047 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413004b070 con 0x55a2b711cb60 2024-03-18T15:46:38.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.856+0000 7f4123fff640 1 -- 172.21.15.96:0/2854204047 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130005040 con 0x55a2b711cb60 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- 172.21.15.96:0/2854204047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 msgr2=0x55a2b7116070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f413002cfd0 tx=0x7f4130005fd0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- 172.21.15.96:0/2854204047 shutdown_connections 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b7121cd0 0x55a2b71165b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711cb60 0x55a2b7116070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- 172.21.15.96:0/2854204047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b711c170 0x55a2b711b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- 172.21.15.96:0/2854204047 >> 172.21.15.96:0/2854204047 conn(0x55a2b71176e0 msgr2=0x55a2b711a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- 172.21.15.96:0/2854204047 shutdown_connections 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- 172.21.15.96:0/2854204047 wait complete. 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- start start 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 Processor -- start 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711c170 0x55a2b712cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b711cb60 0x55a2b712d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a9260 con 0x55a2b7121cd0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a8b60 con 0x55a2b711c170 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a5ce0 con 0x55a2b711cb60 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f4143a83740 10 client.0 _ll_register_callbacks cb 0x7f4134009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f413ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f413a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711c170 0x55a2b712cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f413ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43594/0 (socket says 172.21.15.96:43594) 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.857+0000 7f413ad9d640 1 -- 172.21.15.96:0/546675268 learned_addr learned my addr 172.21.15.96:0/546675268 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4139d9b640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b711cb60 0x55a2b712d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 -- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b711cb60 msgr2=0x55a2b712d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b711cb60 0x55a2b712d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 -- 172.21.15.96:0/546675268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711c170 msgr2=0x55a2b712cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711c170 0x55a2b712cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 -- 172.21.15.96:0/546675268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b70489d0 con 0x55a2b7121cd0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f413ad9d640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f41340133a0 tx=0x7f4134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efad6f6b09384a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:38.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134010b80 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f4134010d20 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413004b070 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4134032030 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4143a83740 10 client.17130 fetch_fsmap learned FSMap version 36 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a2b7199a20 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.858+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413004b070 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8374+0+0 (secure 0 0 0) 0x7f413401f040 con 0x55a2b7121cd0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 10 client.17130 fetch_fsmap finished waiting for FSMap version 36 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 20 client.17130 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b719a480 0x55a2b719c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 4 client.17130 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2b7088d90 con 0x55a2b719a480 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f413a59c640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b719a480 0x55a2b719c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:38.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f413a59c640 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b719a480 0x55a2b719c8a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55cfba5e1a92526 server_cookie=7864b8fb57c7eef0 in_seq=0 out_seq=0 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4121ffb640 10 client.17130 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4121ffb640 1 -- 172.21.15.96:0/546675268 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a2b7088d90 con 0x55a2b719a480 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4121ffb640 10 client.17130 handle_command_reply: tid=0 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.859+0000 7f4143a83740 1 client.17130 shutdown 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 msgr2=0x55a2b7198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f41340133a0 tx=0x7f4134004a40 comp rx=0 tx=0).stop 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 shutdown_connections 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a2b719a480 0x55a2b719c8a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2b7121cd0 0x55a2b7198280 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a2b711cb60 0x55a2b712d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4121ffb640 0 client.17130 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 --2- 172.21.15.96:0/546675268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a2b711c170 0x55a2b712cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 >> 172.21.15.96:0/546675268 conn(0x55a2b71176e0 msgr2=0x55a2b711b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 shutdown_connections 2024-03-18T15:46:38.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 1 -- 172.21.15.96:0/546675268 wait complete. 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4143a83740 20 client.17130 trim_cache size 0 max 16384 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "8e2b9d5c-e623-4a24-ae76-a771d573bdf2": { 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "8e2b9d5c-e623-4a24-ae76-a771d573bdf2", 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:46:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 msgr2=0x7f41240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 0x7f41240a4cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55a2b711d3c0 tx=0x7f4134004ab0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 msgr2=0x7f413c196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f412c00e9c0 tx=0x7f412c00ee80 comp rx=0 tx=0).stop 2024-03-18T15:46:38.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 shutdown_connections 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f41240a2800 0x7f41240a4cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f413c102290 0x7f413c19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f413c1018d0 0x7f413c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 --2- 172.21.15.96:0/614711352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f413c1006e0 0x7f413c196a00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 >> 172.21.15.96:0/614711352 conn(0x7f413c0fbc70 msgr2=0x7f413c104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 shutdown_connections 2024-03-18T15:46:38.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:46:38.860+0000 7f4140e97640 1 -- 172.21.15.96:0/614711352 wait complete. 2024-03-18T15:46:38.871 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:38 smithi096 ceph-mon[31930]: pgmap v548: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 735 KiB/s wr, 305 op/s 2024-03-18T15:46:38.872 INFO:tasks.cephfs.filesystem:scrub status for tag:8e2b9d5c-e623-4a24-ae76-a771d573bdf2 - {'path': '/', 'tag': '8e2b9d5c-e623-4a24-ae76-a771d573bdf2', 'options': 'recursive,force'} 2024-03-18T15:46:39.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:38 smithi137 ceph-mon[35868]: pgmap v548: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 735 KiB/s wr, 305 op/s 2024-03-18T15:46:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:38 smithi193 ceph-mon[36702]: pgmap v548: 97 pgs: 97 active+clean; 682 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 735 KiB/s wr, 305 op/s 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:40.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:39 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:40.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:39 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:40.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:39 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: pgmap v549: 97 pgs: 97 active+clean; 670 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:41.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:40 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: pgmap v549: 97 pgs: 97 active+clean; 670 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:46:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:41.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:40 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: pgmap v549: 97 pgs: 97 active+clean; 670 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:41.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:40 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:43.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:42 smithi137 ceph-mon[35868]: pgmap v550: 97 pgs: 97 active+clean; 665 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 402 op/s 2024-03-18T15:46:43.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:42 smithi193 ceph-mon[36702]: pgmap v550: 97 pgs: 97 active+clean; 665 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 402 op/s 2024-03-18T15:46:43.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:42 smithi096 ceph-mon[31930]: pgmap v550: 97 pgs: 97 active+clean; 665 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 402 op/s 2024-03-18T15:46:45.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:44 smithi137 ceph-mon[35868]: pgmap v551: 97 pgs: 97 active+clean; 660 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 589 op/s 2024-03-18T15:46:45.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:44 smithi193 ceph-mon[36702]: pgmap v551: 97 pgs: 97 active+clean; 660 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 589 op/s 2024-03-18T15:46:45.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:44 smithi096 ceph-mon[31930]: pgmap v551: 97 pgs: 97 active+clean; 660 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 589 op/s 2024-03-18T15:46:47.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:46 smithi137 ceph-mon[35868]: pgmap v552: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 KiB/s wr, 802 op/s 2024-03-18T15:46:47.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:46 smithi193 ceph-mon[36702]: pgmap v552: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 KiB/s wr, 802 op/s 2024-03-18T15:46:47.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:46 smithi096 ceph-mon[31930]: pgmap v552: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 KiB/s wr, 802 op/s 2024-03-18T15:46:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 -- 172.21.15.96:0/2687847421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 msgr2=0x7f5be4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 --2- 172.21.15.96:0/2687847421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4100b20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009900 tx=0x7f5bd402bc30 comp rx=0 tx=0).stop 2024-03-18T15:46:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 -- 172.21.15.96:0/2687847421 shutdown_connections 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 --2- 172.21.15.96:0/2687847421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41022b0 0x7f5be4108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 --2- 172.21.15.96:0/2687847421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41018f0 0x7f5be4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 --2- 172.21.15.96:0/2687847421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4100b20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 -- 172.21.15.96:0/2687847421 >> 172.21.15.96:0/2687847421 conn(0x7f5be40fbc70 msgr2=0x7f5be40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 -- 172.21.15.96:0/2687847421 shutdown_connections 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.927+0000 7f5be9153640 1 -- 172.21.15.96:0/2687847421 wait complete. 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 Processor -- start 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 -- start start 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41018f0 0x7f5be419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41022b0 0x7f5be41a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4069dc0 con 0x7f5be4100700 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4069c40 con 0x7f5be41018f0 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be9153640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5be410b410 con 0x7f5be41022b0 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35706/0 (socket says 172.21.15.96:35706) 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 -- 172.21.15.96:0/95742283 learned_addr learned my addr 172.21.15.96:0/95742283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be359e640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41022b0 0x7f5be41a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41018f0 0x7f5be419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 -- 172.21.15.96:0/95742283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41022b0 msgr2=0x7f5be41a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41022b0 0x7f5be41a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 -- 172.21.15.96:0/95742283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41018f0 msgr2=0x7f5be419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41018f0 0x7f5be419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.928+0000 7f5be2d9d640 1 -- 172.21.15.96:0/95742283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a1910 con 0x7f5be4100700 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be0d99640 1 -- 172.21.15.96:0/95742283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4041070 con 0x7f5be4100700 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a1ba0 con 0x7f5be4100700 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be0d99640 1 -- 172.21.15.96:0/95742283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd4004450 con 0x7f5be4100700 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be0d99640 1 -- 172.21.15.96:0/95742283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4034440 con 0x7f5be4100700 2024-03-18T15:46:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.929+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a2090 con 0x7f5be4100700 2024-03-18T15:46:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be0d99640 1 -- 172.21.15.96:0/95742283 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5bd4044070 con 0x7f5be4100700 2024-03-18T15:46:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 Processor -- start 2024-03-18T15:46:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 -- start start 2024-03-18T15:46:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d52201760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52209330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d52166a40 con 0x560d52201d30 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d5210f920 con 0x560d52201340 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bebe19740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d52162ae0 con 0x560d52206ea0 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35710/0 (socket says 172.21.15.96:35710) 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be2d9d640 1 -- 172.21.15.96:0/3927927137 learned_addr learned my addr 172.21.15.96:0/3927927137 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be359e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d52201760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52209330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.932+0000 7f5be0d99640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 0x7f5bd00a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 -- 172.21.15.96:0/3927927137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 msgr2=0x560d52209330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52209330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 -- 172.21.15.96:0/3927927137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 msgr2=0x560d52201760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d52201760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 -- 172.21.15.96:0/3927927137 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d52127460 con 0x560d52201d30 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be359e640 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d52201760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be359e640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 0x7f5bd00a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be0d99640 1 -- 172.21.15.96:0/95742283 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5bd40f05e0 con 0x7f5be4100700 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5bd403f880 tx=0x7f5bd404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215cccd3a1a41528 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5be359e640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 0x7f5bd00a4d60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x560d52202590 tx=0x7f5bdc004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5bd97fa640 1 -- 172.21.15.96:0/3927927137 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd404b070 con 0x560d52201d30 2024-03-18T15:46:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5bd97fa640 1 -- 172.21.15.96:0/3927927137 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd4004c80 con 0x560d52201d30 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.933+0000 7f5bd97fa640 1 -- 172.21.15.96:0/3927927137 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40bcb40 con 0x560d52201d30 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- 172.21.15.96:0/3927927137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 msgr2=0x560d52206830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5bd403f880 tx=0x7f5bd404d000 comp rx=0 tx=0).stop 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- 172.21.15.96:0/3927927137 shutdown_connections 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52209330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d52206830 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3927927137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d52201760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- 172.21.15.96:0/3927927137 >> 172.21.15.96:0/3927927137 conn(0x560d521fc8b0 msgr2=0x560d521fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- 172.21.15.96:0/3927927137 shutdown_connections 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- 172.21.15.96:0/3927927137 wait complete. 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 -- start start 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.934+0000 7f5bebe19740 1 Processor -- start 2024-03-18T15:46:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d5220b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52201d30 0x560d5220b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560d52209c10 con 0x560d52206ea0 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560d52122d70 con 0x560d52201340 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d52209d90 con 0x560d52201d30 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 10 client.0 _ll_register_callbacks cb 0x560d51e7aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d5220b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35716/0 (socket says 172.21.15.96:35716) 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 -- 172.21.15.96:0/3912316348 learned_addr learned my addr 172.21.15.96:0/3912316348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52201d30 0x560d5220b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 -- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52201d30 msgr2=0x560d5220b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52201d30 0x560d5220b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 -- 172.21.15.96:0/3912316348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 msgr2=0x560d5220b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d5220b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 -- 172.21.15.96:0/3912316348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d51e7b210 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5be359e640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0209a0 tx=0x7f5bdc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fff27e8312fa04f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc010bc0 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5bdc010d60 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.935+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd4004c80 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc029440 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 10 client.17142 fetch_fsmap learned FSMap version 38 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560d51f47620 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd4004c80 con 0x560d52206ea0 2024-03-18T15:46:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f5bdc032070 con 0x560d52206ea0 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 10 client.17142 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 10 client.17142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 20 client.17142 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d52217160 0x560d522195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 4 client.17142 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d52173f80 con 0x560d52217160 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.936+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d52217160 0x560d522195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.937+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d52217160 0x560d522195d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91033f92947ab823 server_cookie=425c86c087b7920 in_seq=0 out_seq=0 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.937+0000 7f5bca7fc640 10 client.17142 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bca7fc640 1 -- 172.21.15.96:0/3912316348 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560d52173f80 con 0x560d52217160 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bca7fc640 10 client.17142 handle_command_reply: tid=0 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 client.17142 shutdown 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 msgr2=0x560d52212800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0209a0 tx=0x7f5bdc004ab0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 shutdown_connections 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d52217160 0x560d522195d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52206ea0 0x560d52212800 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52201d30 0x560d5220b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bca7fc640 0 client.17142 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 --2- 172.21.15.96:0/3912316348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d5220b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 >> 172.21.15.96:0/3912316348 conn(0x560d521fc8b0 msgr2=0x560d521fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 shutdown_connections 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 1 -- 172.21.15.96:0/3912316348 wait complete. 2024-03-18T15:46:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.938+0000 7f5bebe19740 20 client.17142 trim_cache size 0 max 16384 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.953+0000 7f5bebe19740 1 Processor -- start 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.953+0000 7f5bebe19740 1 -- start start 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d521fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d521ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41aaf50 con 0x560d52201d30 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41acb50 con 0x560d52201340 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bebe19740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ab650 con 0x560d52206ea0 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35724/0 (socket says 172.21.15.96:35724) 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 -- 172.21.15.96:0/1883463837 learned_addr learned my addr 172.21.15.96:0/1883463837 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:47.960 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d521fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5be359e640 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d521ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 -- 172.21.15.96:0/1883463837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 msgr2=0x560d521ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d521ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 -- 172.21.15.96:0/1883463837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 msgr2=0x560d521fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d521fcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 -- 172.21.15.96:0/1883463837 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d521ee140 con 0x560d52201d30 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009e60 tx=0x7f5bcc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd047225e9cfeb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bd8ff9640 1 -- 172.21.15.96:0/1883463837 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc022070 con 0x560d52201d30 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bd8ff9640 1 -- 172.21.15.96:0/1883463837 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc0035f0 con 0x560d52201d30 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.954+0000 7f5bd8ff9640 1 -- 172.21.15.96:0/1883463837 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc01d6c0 con 0x560d52201d30 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 -- 172.21.15.96:0/1883463837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 msgr2=0x560d522127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009e60 tx=0x7f5bcc005f90 comp rx=0 tx=0).stop 2024-03-18T15:46:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 -- 172.21.15.96:0/1883463837 shutdown_connections 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d521ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201d30 0x560d522127c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 --2- 172.21.15.96:0/1883463837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201340 0x560d521fcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 -- 172.21.15.96:0/1883463837 >> 172.21.15.96:0/1883463837 conn(0x560d521fc8b0 msgr2=0x560d52206610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 -- 172.21.15.96:0/1883463837 shutdown_connections 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.955+0000 7f5bebe19740 1 -- 172.21.15.96:0/1883463837 wait complete. 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 -- start start 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 Processor -- start 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201d30 0x560d52210ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52211ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41ac480 con 0x560d52201340 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41abd80 con 0x560d52201d30 2024-03-18T15:46:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5be41a8f30 con 0x560d52206ea0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bebe19740 10 client.0 _ll_register_callbacks cb 0x7f5bdc009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35728/0 (socket says 172.21.15.96:35728) 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 -- 172.21.15.96:0/284031724 learned_addr learned my addr 172.21.15.96:0/284031724 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201d30 0x560d52210ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be359e640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52211ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 -- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 msgr2=0x560d52211ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52211ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 -- 172.21.15.96:0/284031724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201d30 msgr2=0x560d52210ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201d30 0x560d52210ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 -- 172.21.15.96:0/284031724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d52212290 con 0x560d52201340 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5be2d9d640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40bcf90 tx=0x7f5bd4003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae69deac178f21d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.956+0000 7f5bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd404b040 con 0x560d52201340 2024-03-18T15:46:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f5bd40bb300 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bcc0035f0 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5bd40a7d10 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 10 client.17148 fetch_fsmap learned FSMap version 38 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560d5227de30 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc0035f0 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7f5bd40b9670 con 0x560d52201340 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 10 client.17148 fetch_fsmap finished waiting for FSMap version 38 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 20 client.17148 populate_metadata read hostname 'smithi096' 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d5227e6a0 0x560d52280ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 4 client.17148 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d5216e2d0 con 0x560d5227e6a0 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.957+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d5227e6a0 0x560d52280ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.958+0000 7f5bdbfff640 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d5227e6a0 0x560d52280ac0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114d5fcc9863aa03 server_cookie=1f6193e780e49942 in_seq=0 out_seq=0 2024-03-18T15:46:47.964 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:47.958+0000 7f5bca7fc640 10 client.17148 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:48.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:47 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:48.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:47 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:48.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:47 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bca7fc640 1 -- 172.21.15.96:0/284031724 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d5216e2d0 con 0x560d5227e6a0 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bca7fc640 10 client.17148 handle_command_reply: tid=0 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 client.17148 shutdown 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 msgr2=0x560d522107a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40bcf90 tx=0x7f5bd4003990 comp rx=0 tx=0).stop 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 shutdown_connections 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d52206ea0 0x560d52211ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560d52201d30 0x560d52210ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560d5227e6a0 0x560d52280ac0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 --2- 172.21.15.96:0/284031724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560d52201340 0x560d522107a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bca7fc640 0 client.17148 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 >> 172.21.15.96:0/284031724 conn(0x560d521fc8b0 msgr2=0x560d522061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 shutdown_connections 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 1 -- 172.21.15.96:0/284031724 wait complete. 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.328+0000 7f5bebe19740 20 client.17148 trim_cache size 0 max 16384 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:46:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:46:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 msgr2=0x7f5bd00a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 0x7f5bd00a4d60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x560d52202590 tx=0x7f5bdc004a40 comp rx=0 tx=0).stop 2024-03-18T15:46:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 msgr2=0x7f5be4199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd402c640 comp rx=0 tx=0).stop 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 shutdown_connections 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f5bd00a28a0 0x7f5bd00a4d60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5be41022b0 0x7f5be41a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5be41018f0 0x7f5be419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 --2- 172.21.15.96:0/95742283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5be4100700 0x7f5be4199d00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 >> 172.21.15.96:0/95742283 conn(0x7f5be40fbc70 msgr2=0x7f5be40fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 shutdown_connections 2024-03-18T15:46:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:46:48.329+0000 7f5be9153640 1 -- 172.21.15.96:0/95742283 wait complete. 2024-03-18T15:46:49.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-mon[35868]: pgmap v553: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 791 KiB/s wr, 701 op/s 2024-03-18T15:46:49.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-mon[35868]: daemon mds.e restarted 2024-03-18T15:46:49.106 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: ignoring --setuser ceph since I am not root 2024-03-18T15:46:49.106 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: ignoring --setgroup ceph since I am not root 2024-03-18T15:46:49.106 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-mds[65687]: -- 172.21.15.137:0/465049602 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5715e9a40 con 0x55b5715bcc00 2024-03-18T15:46:49.106 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:46:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: starting mds.e at 2024-03-18T15:46:49.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:48 smithi193 ceph-mon[36702]: pgmap v553: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 791 KiB/s wr, 701 op/s 2024-03-18T15:46:49.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:48 smithi193 ceph-mon[36702]: daemon mds.e restarted 2024-03-18T15:46:49.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:48 smithi096 ceph-mon[31930]: pgmap v553: 97 pgs: 97 active+clean; 654 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 791 KiB/s wr, 701 op/s 2024-03-18T15:46:49.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:48 smithi096 ceph-mon[31930]: daemon mds.e restarted 2024-03-18T15:46:50.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T15:46:50.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6834/1669469809,v1:172.21.15.137:6835/1669469809] up:boot 2024-03-18T15:46:50.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T15:46:50.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:46:50.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:50.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:49 smithi137 ceph-mon[35868]: pgmap v555: 97 pgs: 97 active+clean; 645 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1020 KiB/s wr, 933 op/s 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6834/1669469809,v1:172.21.15.137:6835/1669469809] up:boot 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:49 smithi193 ceph-mon[36702]: pgmap v555: 97 pgs: 97 active+clean; 645 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1020 KiB/s wr, 933 op/s 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: osdmap e91: 12 total, 12 up, 12 in 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6834/1669469809,v1:172.21.15.137:6835/1669469809] up:boot 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:46:50.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:49 smithi096 ceph-mon[31930]: pgmap v555: 97 pgs: 97 active+clean; 645 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1020 KiB/s wr, 933 op/s 2024-03-18T15:46:52.888 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:52 smithi096 ceph-mon[31930]: pgmap v556: 97 pgs: 97 active+clean; 638 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 933 op/s 2024-03-18T15:46:52.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:52 smithi193 ceph-mon[36702]: pgmap v556: 97 pgs: 97 active+clean; 638 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 933 op/s 2024-03-18T15:46:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:52 smithi137 ceph-mon[35868]: pgmap v556: 97 pgs: 97 active+clean; 638 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 933 op/s 2024-03-18T15:46:54.907 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:54 smithi137 ceph-mon[35868]: pgmap v557: 97 pgs: 97 active+clean; 632 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 770 op/s 2024-03-18T15:46:54.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:54 smithi193 ceph-mon[36702]: pgmap v557: 97 pgs: 97 active+clean; 632 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 770 op/s 2024-03-18T15:46:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:54 smithi096 ceph-mon[31930]: pgmap v557: 97 pgs: 97 active+clean; 632 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 770 op/s 2024-03-18T15:46:56.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:56 smithi193 ceph-mon[36702]: pgmap v558: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:46:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:56 smithi096 ceph-mon[31930]: pgmap v558: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:46:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:56 smithi137 ceph-mon[35868]: pgmap v558: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:46:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:58 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:58.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:46:58 smithi193 ceph-mon[36702]: pgmap v559: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:46:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:58 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:46:58 smithi096 ceph-mon[31930]: pgmap v559: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:46:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:58 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:46:59.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:46:58 smithi137 ceph-mon[35868]: pgmap v559: 97 pgs: 97 active+clean; 624 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 572 op/s 2024-03-18T15:47:00.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:00 smithi193 ceph-mon[36702]: pgmap v560: 97 pgs: 97 active+clean; 591 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-03-18T15:47:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:00 smithi096 ceph-mon[31930]: pgmap v560: 97 pgs: 97 active+clean; 591 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-03-18T15:47:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:00 smithi137 ceph-mon[35868]: pgmap v560: 97 pgs: 97 active+clean; 591 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 586 op/s 2024-03-18T15:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2244985287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088101810 msgr2=0x7fc088101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2244985287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088101810 0x7fc088101c30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc084009900 tx=0x7fc08402bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2244985287 shutdown_connections 2024-03-18T15:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2244985287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0881033c0 0x7fc088109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2244985287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc088102a00 0x7fc088102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2244985287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088101810 0x7fc088101c30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2244985287 >> 172.21.15.96:0/2244985287 conn(0x7fc0880fcda0 msgr2=0x7fc0880ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2244985287 shutdown_connections 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.426+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2244985287 wait complete. 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 Processor -- start 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 -- start start 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 0x7fc088197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0881033c0 0x7fc08819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810d490 con 0x7fc088102a00 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810d310 con 0x7fc0881033c0 2024-03-18T15:47:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08f4e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc08810d610 con 0x7fc088101810 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60090/0 (socket says 172.21.15.96:60090) 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 -- 172.21.15.96:0/2749091602 learned_addr learned my addr 172.21.15.96:0/2749091602 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08da85640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0881033c0 0x7fc08819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08d284640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 0x7fc088197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 -- 172.21.15.96:0/2749091602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 msgr2=0x7fc088197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 0x7fc088197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 -- 172.21.15.96:0/2749091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0881033c0 msgr2=0x7fc08819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0881033c0 0x7fc08819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 -- 172.21.15.96:0/2749091602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08819f730 con 0x7fc088102a00 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08d284640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 0x7fc088197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc08ca83640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d950 tx=0x7fc07800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.427+0000 7fc07e7fc640 1 -- 172.21.15.96:0/2749091602 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078017030 con 0x7fc088102a00 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.428+0000 7fc07e7fc640 1 -- 172.21.15.96:0/2749091602 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc078004020 con 0x7fc088102a00 2024-03-18T15:47:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.428+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08819fa20 con 0x7fc088102a00 2024-03-18T15:47:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.428+0000 7fc07e7fc640 1 -- 172.21.15.96:0/2749091602 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0780047a0 con 0x7fc088102a00 2024-03-18T15:47:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.428+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08819ff70 con 0x7fc088102a00 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc07e7fc640 1 -- 172.21.15.96:0/2749091602 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc0780041d0 con 0x7fc088102a00 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 Processor -- start 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc07e7fc640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 0x7fc0740a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 -- start start 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc08da85640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 0x7fc0740a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac527830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc07e7fc640 1 -- 172.21.15.96:0/2749091602 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc0780ce470 con 0x7fc088102a00 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac52a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efac487a40 con 0x55efac522340 2024-03-18T15:47:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55efac430920 con 0x55efac527ea0 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.430+0000 7fc0921ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efac483ae0 con 0x55efac522d30 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac527830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60096/0 (socket says 172.21.15.96:60096) 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac527830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:48502/0 (socket says 172.21.15.96:48502) 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 -- 172.21.15.96:0/1186113112 learned_addr learned my addr 172.21.15.96:0/1186113112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08da85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac52a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08da85640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 0x7fc0740a4d60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc080014b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 -- 172.21.15.96:0/1186113112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 msgr2=0x55efac527830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac527830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 -- 172.21.15.96:0/1186113112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 msgr2=0x55efac52a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac52a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 -- 172.21.15.96:0/1186113112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efac448460 con 0x55efac522340 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08da85640 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac52a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:01.442 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc08d284640 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc084009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251c90efd86d1d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc0737fe640 1 -- 172.21.15.96:0/1186113112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402cc40 con 0x55efac522340 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc0737fe640 1 -- 172.21.15.96:0/1186113112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc084005040 con 0x55efac522340 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.431+0000 7fc0737fe640 1 -- 172.21.15.96:0/1186113112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084004900 con 0x55efac522340 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- 172.21.15.96:0/1186113112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 msgr2=0x55efac522760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc084000c00 tx=0x7fc084009660 comp rx=0 tx=0).stop 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- 172.21.15.96:0/1186113112 shutdown_connections 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac52a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac527830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1186113112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac522760 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- 172.21.15.96:0/1186113112 >> 172.21.15.96:0/1186113112 conn(0x55efac51d8b0 msgr2=0x55efac51dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- 172.21.15.96:0/1186113112 shutdown_connections 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- 172.21.15.96:0/1186113112 wait complete. 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 -- start start 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.432+0000 7fc0921ac740 1 Processor -- start 2024-03-18T15:47:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac522340 0x55efac52c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 0x55efac533b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55efac52adf0 con 0x55efac522d30 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55efac443d70 con 0x55efac522340 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efac52af70 con 0x55efac527ea0 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 10 client.0 _ll_register_callbacks cb 0x55efac19ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60106/0 (socket says 172.21.15.96:60106) 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 -- 172.21.15.96:0/1757573000 learned_addr learned my addr 172.21.15.96:0/1757573000 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08d284640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac522340 0x55efac52c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08da85640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 0x55efac533b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 -- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 msgr2=0x55efac533b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 0x55efac533b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 -- 172.21.15.96:0/1757573000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac522340 msgr2=0x55efac52c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac522340 0x55efac52c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 -- 172.21.15.96:0/1757573000 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efac19c210 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08da85640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 0x55efac533b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc08ca83640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc0780cc2b0 tx=0x7fc07809b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed35e5fd26738ae server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0780b8060 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc0780a30f0 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.433+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc084005040 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0780a3580 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 10 client.17166 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55efac268620 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc084005040 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fc078099ca0 con 0x55efac522d30 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 10 client.17166 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 20 client.17166 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac538450 0x55efac53a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 4 client.17166 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efac494f80 con 0x55efac538450 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.434+0000 7fc08d284640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac538450 0x55efac53a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.435+0000 7fc070ff9640 10 client.17166 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.435+0000 7fc08d284640 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac538450 0x55efac53a840 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ec45c7ce0192188 server_cookie=7ba476746629a23f in_seq=0 out_seq=0 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc070ff9640 1 -- 172.21.15.96:0/1757573000 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55efac494f80 con 0x55efac538450 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc070ff9640 10 client.17166 handle_command_reply: tid=0 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 client.17166 shutdown 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 msgr2=0x55efac52cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc0780cc2b0 tx=0x7fc07809b610 comp rx=0 tx=0).stop 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 shutdown_connections 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac527ea0 0x55efac533b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac538450 0x55efac53a840 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac52cb10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 --2- 172.21.15.96:0/1757573000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac522340 0x55efac52c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc070ff9640 0 client.17166 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 >> 172.21.15.96:0/1757573000 conn(0x55efac51d8b0 msgr2=0x55efac51cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 shutdown_connections 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 1 -- 172.21.15.96:0/1757573000 wait complete. 2024-03-18T15:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.436+0000 7fc0921ac740 20 client.17166 trim_cache size 0 max 16384 2024-03-18T15:47:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 Processor -- start 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 -- start start 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac533b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac521220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881a8d80 con 0x55efac522340 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881aa920 con 0x55efac527ea0 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc0921ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881a9480 con 0x55efac522d30 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc08d284640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc08da85640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac521220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc08d284640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60116/0 (socket says 172.21.15.96:60116) 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc08d284640 1 -- 172.21.15.96:0/3462197417 learned_addr learned my addr 172.21.15.96:0/3462197417 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.451+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac533b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.454 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 -- 172.21.15.96:0/3462197417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 msgr2=0x55efac533b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac533b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 -- 172.21.15.96:0/3462197417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 msgr2=0x55efac521220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac521220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 -- 172.21.15.96:0/3462197417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efac522720 con 0x55efac522340 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc08d284640 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc08402c170 tx=0x7fc08402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf4262c841f4078 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc072ffd640 1 -- 172.21.15.96:0/3462197417 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08402cd00 con 0x55efac522340 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc072ffd640 1 -- 172.21.15.96:0/3462197417 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc084009520 con 0x55efac522340 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.452+0000 7fc072ffd640 1 -- 172.21.15.96:0/3462197417 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084005040 con 0x55efac522340 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- 172.21.15.96:0/3462197417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 msgr2=0x55efac518a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc08402c170 tx=0x7fc08402e040 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- 172.21.15.96:0/3462197417 shutdown_connections 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac521220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522d30 0x55efac533b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3462197417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522340 0x55efac518a40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.455 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- 172.21.15.96:0/3462197417 >> 172.21.15.96:0/3462197417 conn(0x55efac51d8b0 msgr2=0x55efac520b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- 172.21.15.96:0/3462197417 shutdown_connections 2024-03-18T15:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- 172.21.15.96:0/3462197417 wait complete. 2024-03-18T15:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- start start 2024-03-18T15:47:01.456 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 Processor -- start 2024-03-18T15:47:01.458 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522340 0x55efac531b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac533090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881aa250 con 0x55efac522d30 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881a9b80 con 0x55efac527ea0 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881a6d60 con 0x55efac522340 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc0921ac740 10 client.0 _ll_register_callbacks cb 0x7fc07809a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc08ca83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60130/0 (socket says 172.21.15.96:60130) 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc08d284640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522340 0x55efac531b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc08ca83640 1 -- 172.21.15.96:0/3667543329 learned_addr learned my addr 172.21.15.96:0/3667543329 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.453+0000 7fc08da85640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac533090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 -- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522340 msgr2=0x55efac531b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522340 0x55efac531b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 -- 172.21.15.96:0/3667543329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 msgr2=0x55efac533090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac533090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 -- 172.21.15.96:0/3667543329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efac533680 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc08ca83640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc07809a060 tx=0x7fc0780045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72329f4f6a2978dd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078014070 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc07809a430 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc084009520 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0780b8040 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc0921ac740 10 client.17172 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55efac59f0a0 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc084009520 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.454+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7fc078009a30 con 0x55efac522d30 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 10 client.17172 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 10 client.17172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 20 client.17172 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:01.460 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac59faa0 0x55efac5a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 4 client.17172 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:47:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efac48f2d0 con 0x55efac59faa0 2024-03-18T15:47:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc08d284640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac59faa0 0x55efac5a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc08d284640 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac59faa0 0x55efac5a1ec0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be962d66f0a7a575 server_cookie=fa7e23c974ca47d3 in_seq=0 out_seq=0 2024-03-18T15:47:01.461 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.455+0000 7fc070ff9640 10 client.17172 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.472+0000 7fc070ff9640 1 -- 172.21.15.96:0/3667543329 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efac48f2d0 con 0x55efac59faa0 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.472+0000 7fc070ff9640 10 client.17172 handle_command_reply: tid=0 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.472+0000 7fc0921ac740 1 client.17172 shutdown 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 msgr2=0x55efac532080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc07809a060 tx=0x7fc0780045b0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 shutdown_connections 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efac527ea0 0x55efac533090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55efac59faa0 0x55efac5a1ec0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55efac522d30 0x55efac532080 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 --2- 172.21.15.96:0/3667543329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efac522340 0x55efac531b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc070ff9640 0 client.17172 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 >> 172.21.15.96:0/3667543329 conn(0x55efac51d8b0 msgr2=0x55efac5213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 shutdown_connections 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 1 -- 172.21.15.96:0/3667543329 wait complete. 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc0921ac740 20 client.17172 trim_cache size 0 max 16384 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:47:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 msgr2=0x7fc0740a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 0x7fc0740a4d60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc080009fd0 tx=0x7fc080014b70 comp rx=0 tx=0).stop 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 msgr2=0x7fc088198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.473+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d950 tx=0x7fc07800de10 comp rx=0 tx=0).stop 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 shutdown_connections 2024-03-18T15:47:01.476 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0740a28a0 0x7fc0740a4d60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0881033c0 0x7fc08819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc088102a00 0x7fc088198050 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 --2- 172.21.15.96:0/2749091602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc088101810 0x7fc088197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 >> 172.21.15.96:0/2749091602 conn(0x7fc0880fcda0 msgr2=0x7fc088105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 shutdown_connections 2024-03-18T15:47:01.477 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:01.474+0000 7fc08f4e8640 1 -- 172.21.15.96:0/2749091602 wait complete. 2024-03-18T15:47:02.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:02 smithi193 ceph-mon[36702]: pgmap v561: 97 pgs: 97 active+clean; 579 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 473 op/s 2024-03-18T15:47:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:02 smithi096 ceph-mon[31930]: pgmap v561: 97 pgs: 97 active+clean; 579 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 473 op/s 2024-03-18T15:47:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:02 smithi137 ceph-mon[35868]: pgmap v561: 97 pgs: 97 active+clean; 579 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 473 op/s 2024-03-18T15:47:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:04 smithi193 ceph-mon[36702]: pgmap v562: 97 pgs: 97 active+clean; 575 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 483 op/s 2024-03-18T15:47:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:04 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:04.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:04 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:04 smithi096 ceph-mon[31930]: pgmap v562: 97 pgs: 97 active+clean; 575 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 483 op/s 2024-03-18T15:47:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:04 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:04.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:04 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:04 smithi137 ceph-mon[35868]: pgmap v562: 97 pgs: 97 active+clean; 575 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 483 op/s 2024-03-18T15:47:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:04 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:05.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:04 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:05 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:05 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:05.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:05 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:05 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:05 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:05.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:05 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:06.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:05 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:06.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:05 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:06.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:05 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:06.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:06 smithi193 ceph-mon[36702]: pgmap v563: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 507 op/s 2024-03-18T15:47:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:06 smithi096 ceph-mon[31930]: pgmap v563: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 507 op/s 2024-03-18T15:47:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:06 smithi137 ceph-mon[35868]: pgmap v563: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 507 op/s 2024-03-18T15:47:08.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:47:08.874 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:47:08.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:08 smithi193 ceph-mon[36702]: pgmap v564: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 850 KiB/s wr, 398 op/s 2024-03-18T15:47:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:08 smithi096 ceph-mon[31930]: pgmap v564: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 850 KiB/s wr, 398 op/s 2024-03-18T15:47:08.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.978+0000 7f44f477e640 1 -- 172.21.15.96:0/741807665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec102290 msgr2=0x7f44ec108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.978+0000 7f44f477e640 1 --2- 172.21.15.96:0/741807665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec102290 0x7f44ec108b20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009ca0 tx=0x7f44e402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- 172.21.15.96:0/741807665 shutdown_connections 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- 172.21.15.96:0/741807665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec102290 0x7f44ec108b20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- 172.21.15.96:0/741807665 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 0x7f44ec101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- 172.21.15.96:0/741807665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec1006e0 0x7f44ec100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- 172.21.15.96:0/741807665 >> 172.21.15.96:0/741807665 conn(0x7f44ec0fbc70 msgr2=0x7f44ec0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- 172.21.15.96:0/741807665 shutdown_connections 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- 172.21.15.96:0/741807665 wait complete. 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 Processor -- start 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- start start 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 0x7f44ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec102290 0x7f44ec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec10d480 con 0x7f44ec1006e0 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec10d300 con 0x7f44ec1018d0 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f477e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec10d600 con 0x7f44ec102290 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f251a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f251a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55996/0 (socket says 172.21.15.96:55996) 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.979+0000 7f44f251a640 1 -- 172.21.15.96:0/4120210182 learned_addr learned my addr 172.21.15.96:0/4120210182 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec102290 0x7f44ec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f1d19640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 0x7f44ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 -- 172.21.15.96:0/4120210182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec102290 msgr2=0x7f44ec19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec102290 0x7f44ec19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 -- 172.21.15.96:0/4120210182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 msgr2=0x7f44ec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 0x7f44ec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 -- 172.21.15.96:0/4120210182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec19f6d0 con 0x7f44ec1006e0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f251a640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f44e000e9c0 tx=0x7f44e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44df7fe640 1 -- 172.21.15.96:0/4120210182 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e000cd80 con 0x7f44ec1006e0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec19f9c0 con 0x7f44ec1006e0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44df7fe640 1 -- 172.21.15.96:0/4120210182 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e00035f0 con 0x7f44ec1006e0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44df7fe640 1 -- 172.21.15.96:0/4120210182 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0010690 con 0x7f44ec1006e0 2024-03-18T15:47:08.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.980+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec19fec0 con 0x7f44ec1006e0 2024-03-18T15:47:08.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44df7fe640 1 -- 172.21.15.96:0/4120210182 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f44e00037a0 con 0x7f44ec1006e0 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 Processor -- start 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 -- start start 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c10590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c18160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564700b75500 con 0x564700c10b60 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564700b1dbe0 con 0x564700c10170 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f736e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564700b715a0 con 0x564700c15cd0 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.983+0000 7f44f251a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56010/0 (socket says 172.21.15.96:56010) 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f2d1b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c10590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 -- 172.21.15.96:0/24266940 learned_addr learned my addr 172.21.15.96:0/24266940 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f1d19640 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c18160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44df7fe640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 0x7f44d80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 0x7f44d80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 -- 172.21.15.96:0/24266940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 msgr2=0x564700c18160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c18160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 -- 172.21.15.96:0/24266940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 msgr2=0x564700c10590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c10590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 -- 172.21.15.96:0/24266940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564700b314b0 con 0x564700c10b60 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f1d19640 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c18160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44df7fe640 1 -- 172.21.15.96:0/4120210182 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44e00cedf0 con 0x7f44ec1006e0 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f251a640 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f44e009b6f0 tx=0x7f44e0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac94052da19cd5e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 0x7f44d80a4cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005e30 tx=0x7f44e402c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44dcff9640 1 -- 172.21.15.96:0/24266940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e009b2a0 con 0x564700c10b60 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44dcff9640 1 -- 172.21.15.96:0/24266940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e009be60 con 0x564700c10b60 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.984+0000 7f44dcff9640 1 -- 172.21.15.96:0/24266940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e009b0c0 con 0x564700c10b60 2024-03-18T15:47:08.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- 172.21.15.96:0/24266940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 msgr2=0x564700c15660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f44e009b6f0 tx=0x7f44e0003f10 comp rx=0 tx=0).stop 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- 172.21.15.96:0/24266940 shutdown_connections 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c18160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c15660 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 --2- 172.21.15.96:0/24266940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c10590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- 172.21.15.96:0/24266940 >> 172.21.15.96:0/24266940 conn(0x564700c0b6e0 msgr2=0x564700c0bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- 172.21.15.96:0/24266940 shutdown_connections 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- 172.21.15.96:0/24266940 wait complete. 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 -- start start 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.985+0000 7f44f736e740 1 Processor -- start 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10b60 0x564700c1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564700c18c20 con 0x564700c15cd0 2024-03-18T15:47:08.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564700b31160 con 0x564700c10170 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564700c18da0 con 0x564700c10b60 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 10 client.0 _ll_register_callbacks cb 0x564700b15150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56020/0 (socket says 172.21.15.96:56020) 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f251a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 -- 172.21.15.96:0/2709656689 learned_addr learned my addr 172.21.15.96:0/2709656689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f1d19640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10b60 0x564700c1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 -- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10b60 msgr2=0x564700c1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10b60 0x564700c1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 -- 172.21.15.96:0/2709656689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 msgr2=0x564700c1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 -- 172.21.15.96:0/2709656689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647009537d0 con 0x564700c15cd0 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f251a640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:08.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f44e4036000 tx=0x7f44e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8822c2d823c3319f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4053060 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44e4004040 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.986+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e009be60 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f44e4047070 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 10 client.17184 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5647008893f0 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e009be60 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f44e403f070 con 0x564700c15cd0 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 10 client.17184 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 20 client.17184 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c26140 0x564700c285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 4 client.17184 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564700b82a40 con 0x564700c26140 2024-03-18T15:47:08.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.987+0000 7f44f251a640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c26140 0x564700c285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.988+0000 7f44d5ffb640 10 client.17184 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.988+0000 7f44f251a640 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c26140 0x564700c285e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b973855e38a6b43c server_cookie=8b3a7be8ac522510 in_seq=0 out_seq=0 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44d5ffb640 1 -- 172.21.15.96:0/2709656689 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564700b82a40 con 0x564700c26140 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44d5ffb640 10 client.17184 handle_command_reply: tid=0 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 client.17184 shutdown 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 msgr2=0x564700c21ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f44e4036000 tx=0x7f44e40095c0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 shutdown_connections 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c26140 0x564700c285e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c21ba0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10b60 0x564700c1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44d5ffb640 0 client.17184 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 --2- 172.21.15.96:0/2709656689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 >> 172.21.15.96:0/2709656689 conn(0x564700c0b6e0 msgr2=0x564700c0a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:08.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 shutdown_connections 2024-03-18T15:47:08.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 1 -- 172.21.15.96:0/2709656689 wait complete. 2024-03-18T15:47:08.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:08.989+0000 7f44f736e740 20 client.17184 trim_cache size 0 max 16384 2024-03-18T15:47:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 Processor -- start 2024-03-18T15:47:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 -- start start 2024-03-18T15:47:09.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c0a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1a8cd0 con 0x564700c10b60 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1aa870 con 0x564700c10170 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.006+0000 7f44f736e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1a93d0 con 0x564700c15cd0 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56028/0 (socket says 172.21.15.96:56028) 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 -- 172.21.15.96:0/468394971 learned_addr learned my addr 172.21.15.96:0/468394971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f251a640 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c0a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 -- 172.21.15.96:0/468394971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 msgr2=0x564700c0a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c0a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 -- 172.21.15.96:0/468394971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 msgr2=0x564700c0a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c0a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 -- 172.21.15.96:0/468394971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647008490f0 con 0x564700c10b60 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44f1d19640 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009e60 tx=0x7f44e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766f27ae9d7185ab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44d7fff640 1 -- 172.21.15.96:0/468394971 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80120d0 con 0x564700c10b60 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44d7fff640 1 -- 172.21.15.96:0/468394971 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8012270 con 0x564700c10b60 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.007+0000 7f44d7fff640 1 -- 172.21.15.96:0/468394971 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8010430 con 0x564700c10b60 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- 172.21.15.96:0/468394971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 msgr2=0x564700c21b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009e60 tx=0x7f44e8005f90 comp rx=0 tx=0).stop 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- 172.21.15.96:0/468394971 shutdown_connections 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c15cd0 0x564700c0a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c10b60 0x564700c21b60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- 172.21.15.96:0/468394971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10170 0x564700c0a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- 172.21.15.96:0/468394971 >> 172.21.15.96:0/468394971 conn(0x564700c0b6e0 msgr2=0x564700c12060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- 172.21.15.96:0/468394971 shutdown_connections 2024-03-18T15:47:09.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- 172.21.15.96:0/468394971 wait complete. 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- start start 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 Processor -- start 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10170 0x564700c1f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10b60 0x564700c1f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1aa1a0 con 0x564700c15cd0 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1a9ad0 con 0x564700c10b60 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec1a6cb0 con 0x564700c10170 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f736e740 10 client.0 _ll_register_callbacks cb 0x7f44e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f2d1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f251a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10170 0x564700c1f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f2d1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56038/0 (socket says 172.21.15.96:56038) 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f2d1b640 1 -- 172.21.15.96:0/952100444 learned_addr learned my addr 172.21.15.96:0/952100444 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.008+0000 7f44f1d19640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10b60 0x564700c1f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 -- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10170 msgr2=0x564700c1f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10170 0x564700c1f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 -- 172.21.15.96:0/952100444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10b60 msgr2=0x564700c1f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10b60 0x564700c1f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 -- 172.21.15.96:0/952100444 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564700b3c9d0 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f2d1b640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f44e40325d0 tx=0x7f44e404eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429345619b143d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4053040 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f44e4034920 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e8012270 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f44e4068030 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 10 client.17190 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564700c21df0 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8012270 con 0x564700c15cd0 2024-03-18T15:47:09.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7f44e403f070 con 0x564700c15cd0 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 10 client.17190 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 20 client.17190 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c8cb30 0x564700c8ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 4 client.17190 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.009+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564700b7cd90 con 0x564700c8cb30 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f251a640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c8cb30 0x564700c8ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44d5ffb640 10 client.17190 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f251a640 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c8cb30 0x564700c8ef50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2270e221b85928cf server_cookie=f4a1c7a4fff0f1c3 in_seq=0 out_seq=0 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44d5ffb640 1 -- 172.21.15.96:0/952100444 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564700b7cd90 con 0x564700c8cb30 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44d5ffb640 10 client.17190 handle_command_reply: tid=0 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 client.17190 shutdown 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 msgr2=0x564700c206c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f44e40325d0 tx=0x7f44e404eba0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 shutdown_connections 2024-03-18T15:47:09.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564700c8cb30 0x564700c8ef50 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564700c15cd0 0x564700c206c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44d5ffb640 0 client.17190 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564700c10b60 0x564700c1f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 --2- 172.21.15.96:0/952100444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564700c10170 0x564700c1f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 >> 172.21.15.96:0/952100444 conn(0x564700c0b6e0 msgr2=0x564700c0e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 shutdown_connections 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.010+0000 7f44f736e740 1 -- 172.21.15.96:0/952100444 wait complete. 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f736e740 20 client.17190 trim_cache size 0 max 16384 2024-03-18T15:47:09.018 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:47:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:47:09.019 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:09.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 msgr2=0x7f44d80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 0x7f44d80a4cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f44e4005e30 tx=0x7f44e402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 msgr2=0x7f44ec197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f44e000e9c0 tx=0x7f44e000ee80 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 shutdown_connections 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f44d80a2800 0x7f44d80a4cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44ec102290 0x7f44ec19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f44ec1018d0 0x7f44ec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 --2- 172.21.15.96:0/4120210182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44ec1006e0 0x7f44ec197ab0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 >> 172.21.15.96:0/4120210182 conn(0x7f44ec0fbc70 msgr2=0x7f44ec104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 shutdown_connections 2024-03-18T15:47:09.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.011+0000 7f44f477e640 1 -- 172.21.15.96:0/4120210182 wait complete. 2024-03-18T15:47:09.022 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:47:09.022 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:47:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:08 smithi137 ceph-mon[35868]: pgmap v564: 97 pgs: 97 active+clean; 563 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 850 KiB/s wr, 398 op/s 2024-03-18T15:47:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 -- 172.21.15.96:0/2620372917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681022a0 msgr2=0x7f8168108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 --2- 172.21.15.96:0/2620372917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681022a0 0x7f8168108b30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8150009860 tx=0x7f815002bb70 comp rx=0 tx=0).stop 2024-03-18T15:47:09.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 -- 172.21.15.96:0/2620372917 shutdown_connections 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 --2- 172.21.15.96:0/2620372917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681022a0 0x7f8168108b30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 --2- 172.21.15.96:0/2620372917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681018e0 0x7f8168101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 --2- 172.21.15.96:0/2620372917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681006f0 0x7f8168100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 -- 172.21.15.96:0/2620372917 >> 172.21.15.96:0/2620372917 conn(0x7f81680fbc60 msgr2=0x7f81680fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 -- 172.21.15.96:0/2620372917 shutdown_connections 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.126+0000 7f816eb32640 1 -- 172.21.15.96:0/2620372917 wait complete. 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 Processor -- start 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 -- start start 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 0x7f816819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f816810c350 con 0x7f81681018e0 2024-03-18T15:47:09.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f816810c1d0 con 0x7f81681006f0 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816eb32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f816810c4d0 con 0x7f81681022a0 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816c8ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f8167fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816c8ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55184/0 (socket says 172.21.15.96:55184) 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816c8ce640 1 -- 172.21.15.96:0/1904483482 learned_addr learned my addr 172.21.15.96:0/1904483482 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f8167fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56060/0 (socket says 172.21.15.96:56060) 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.127+0000 7f816d0cf640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 0x7f816819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 -- 172.21.15.96:0/1904483482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 msgr2=0x7f816819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 0x7f816819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 -- 172.21.15.96:0/1904483482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 msgr2=0x7f816806aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 -- 172.21.15.96:0/1904483482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819f690 con 0x7f81681018e0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f816c8ce640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f816d0cf640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 0x7f816819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8167fff640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f815800eb00 tx=0x7f815800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158010430 con 0x7f81681018e0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816819f980 con 0x7f81681018e0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8158010ab0 con 0x7f81681018e0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158017bb0 con 0x7f81681018e0 2024-03-18T15:47:09.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.128+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816819fed0 con 0x7f81681018e0 2024-03-18T15:47:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.129+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8168107a10 con 0x7f81681018e0 2024-03-18T15:47:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.130+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f8158010c60 con 0x7f81681018e0 2024-03-18T15:47:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.131+0000 7f8165ffb640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 0x7f81540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.131+0000 7f816c8ce640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 0x7f81540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.131+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f81580cf430 con 0x7f81681018e0 2024-03-18T15:47:09.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.131+0000 7f816c8ce640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 0x7f81540a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8160011490 tx=0x7f8160009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.136+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f815809c4a0 con 0x7f81681018e0 2024-03-18T15:47:09.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.357+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81680623e0 con 0x7f81681018e0 2024-03-18T15:47:09.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.358+0000 7f8165ffb640 1 -- 172.21.15.96:0/1904483482 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11941 (secure 0 0 0) 0x7f815809bda0 con 0x7f81681018e0 2024-03-18T15:47:09.359 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:47:09.359 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:46:48.714661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:47:09.359 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 40 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 msgr2=0x7f81540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 0x7f81540a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8160011490 tx=0x7f8160009260 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 msgr2=0x7f816819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f815800eb00 tx=0x7f815800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 shutdown_connections 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f81540a2800 0x7f81540a4cc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81681022a0 0x7f816819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81681018e0 0x7f816819c790 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 --2- 172.21.15.96:0/1904483482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81681006f0 0x7f816806aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 >> 172.21.15.96:0/1904483482 conn(0x7f81680fbc60 msgr2=0x7f8168104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 shutdown_connections 2024-03-18T15:47:09.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.360+0000 7f816eb32640 1 -- 172.21.15.96:0/1904483482 wait complete. 2024-03-18T15:47:09.373 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- 172.21.15.96:0/2990169992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc102290 msgr2=0x7ffbcc108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/2990169992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc102290 0x7ffbcc108b20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009cc0 tx=0x7ffbc402e310 comp rx=0 tx=0).stop 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- 172.21.15.96:0/2990169992 shutdown_connections 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/2990169992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc102290 0x7ffbcc108b20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/2990169992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/2990169992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- 172.21.15.96:0/2990169992 >> 172.21.15.96:0/2990169992 conn(0x7ffbcc0fbc70 msgr2=0x7ffbcc0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- 172.21.15.96:0/2990169992 shutdown_connections 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- 172.21.15.96:0/2990169992 wait complete. 2024-03-18T15:47:09.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 Processor -- start 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 -- start start 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc1958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.504+0000 7ffbd16c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc102290 0x7ffbcc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbd16c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10b260 con 0x7ffbcc1018d0 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbd16c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10b0e0 con 0x7ffbcc1006e0 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbd16c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc10b3e0 con 0x7ffbcc102290 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbcaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc1958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56082/0 (socket says 172.21.15.96:56082) 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 -- 172.21.15.96:0/4056596827 learned_addr learned my addr 172.21.15.96:0/4056596827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc102290 0x7ffbcc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 -- 172.21.15.96:0/4056596827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc102290 msgr2=0x7ffbcc19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc102290 0x7ffbcc19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 -- 172.21.15.96:0/4056596827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 msgr2=0x7ffbcc1958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc1958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 -- 172.21.15.96:0/4056596827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc19d4d0 con 0x7ffbcc1018d0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e9f0 tx=0x7ffbb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc8ff9640 1 -- 172.21.15.96:0/4056596827 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400cd80 con 0x7ffbcc1018d0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc8ff9640 1 -- 172.21.15.96:0/4056596827 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb4010910 con 0x7ffbcc1018d0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc19d7c0 con 0x7ffbcc1018d0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbc8ff9640 1 -- 172.21.15.96:0/4056596827 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4018990 con 0x7ffbcc1018d0 2024-03-18T15:47:09.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.505+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc19dd60 con 0x7ffbcc1018d0 2024-03-18T15:47:09.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.508+0000 7ffbc8ff9640 1 -- 172.21.15.96:0/4056596827 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ffbb4010ac0 con 0x7ffbcc1018d0 2024-03-18T15:47:09.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.508+0000 7ffbc8ff9640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 0x7ffbb80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.508+0000 7ffbd438e740 1 Processor -- start 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 -- start start 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 0x7ffbb80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbc8ff9640 1 -- 172.21.15.96:0/4056596827 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ffbb40ced40 con 0x7ffbcc1018d0 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d987d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5d98a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560a5d8e7490 con 0x560a5d9883b0 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a5d88fc90 con 0x560a5d982850 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbd438e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a5d8e3530 con 0x560a5d983240 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d987d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55208/0 (socket says 172.21.15.96:55208) 2024-03-18T15:47:09.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 -- 172.21.15.96:0/3026954234 learned_addr learned my addr 172.21.15.96:0/3026954234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5d98a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 0x7ffbb80a4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc101730 tx=0x7ffbbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 -- 172.21.15.96:0/3026954234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 msgr2=0x560a5d987d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d987d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 -- 172.21.15.96:0/3026954234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 msgr2=0x560a5d98a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5d98a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.509+0000 7ffbcaffd640 1 -- 172.21.15.96:0/3026954234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5d94a500 con 0x560a5d982850 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.510+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00f670 tx=0x7ffbbc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0eddc04bc808d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.510+0000 7ffbc17fa640 1 -- 172.21.15.96:0/3026954234 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00fe60 con 0x560a5d982850 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.510+0000 7ffbc17fa640 1 -- 172.21.15.96:0/3026954234 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbbc01c8f0 con 0x560a5d982850 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.510+0000 7ffbc17fa640 1 -- 172.21.15.96:0/3026954234 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc025960 con 0x560a5d982850 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- 172.21.15.96:0/3026954234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 msgr2=0x560a5d982c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00f670 tx=0x7ffbbc014000 comp rx=0 tx=0).stop 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- 172.21.15.96:0/3026954234 shutdown_connections 2024-03-18T15:47:09.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5d98a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d987d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3026954234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d982c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- 172.21.15.96:0/3026954234 >> 172.21.15.96:0/3026954234 conn(0x560a5d97ddc0 msgr2=0x560a5d97e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- 172.21.15.96:0/3026954234 shutdown_connections 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- 172.21.15.96:0/3026954234 wait complete. 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- start start 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 Processor -- start 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d98cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d9883b0 0x560a5d9942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560a5d98b360 con 0x560a5d983240 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a5d8a3230 con 0x560a5d982850 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560a5d98b4e0 con 0x560a5d9883b0 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbd438e740 10 client.0 _ll_register_callbacks cb 0x560a5d8df010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56106/0 (socket says 172.21.15.96:56106) 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbc3fff640 1 -- 172.21.15.96:0/2113927151 learned_addr learned my addr 172.21.15.96:0/2113927151 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d98cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.511+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d9883b0 0x560a5d9942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 -- 172.21.15.96:0/2113927151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d9883b0 msgr2=0x560a5d9942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d9883b0 0x560a5d9942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 -- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 msgr2=0x560a5d98cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d98cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 -- 172.21.15.96:0/2113927151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5d8b2100 con 0x560a5d983240 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d98cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cc860 tx=0x7ffbb409b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b610f1bbcc9bdc4a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40b4060 con 0x560a5d983240 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ffbb4097dd0 con 0x560a5d983240 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbbc01c8f0 con 0x560a5d983240 2024-03-18T15:47:09.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbb4018b30 con 0x560a5d983240 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 10 client.17208 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a5d5fb3f0 con 0x560a5d983240 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbbc01c8f0 con 0x560a5d983240 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ffbb40b4200 con 0x560a5d983240 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 10 client.17208 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 10 client.17208 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 20 client.17208 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5d99af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.512+0000 7ffbd438e740 4 client.17208 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.513+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a5d8f49d0 con 0x560a5d998a80 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.513+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5d99af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.513+0000 7ffbb27fc640 10 client.17208 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.513+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5d99af20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c2cbc644957aa2b server_cookie=406b5e91106ed197 in_seq=0 out_seq=0 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbb27fc640 1 -- 172.21.15.96:0/2113927151 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560a5d8f49d0 con 0x560a5d998a80 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbb27fc640 10 client.17208 handle_command_reply: tid=0 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 client.17208 shutdown 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 msgr2=0x560a5d98d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cc860 tx=0x7ffbb409b820 comp rx=0 tx=0).stop 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 shutdown_connections 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5d99af20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbb27fc640 0 client.17208 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d9883b0 0x560a5d9942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d98d2a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 --2- 172.21.15.96:0/2113927151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d98cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 >> 172.21.15.96:0/2113927151 conn(0x560a5d97ddc0 msgr2=0x560a5d97cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 shutdown_connections 2024-03-18T15:47:09.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 1 -- 172.21.15.96:0/2113927151 wait complete. 2024-03-18T15:47:09.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.514+0000 7ffbd438e740 20 client.17208 trim_cache size 0 max 16384 2024-03-18T15:47:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 Processor -- start 2024-03-18T15:47:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 -- start start 2024-03-18T15:47:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d982850 0x560a5d984740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 0x560a5d97c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a6c30 con 0x560a5d983240 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a87d0 con 0x560a5d9883b0 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.527+0000 7ffbd438e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a7330 con 0x560a5d982850 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56116/0 (socket says 172.21.15.96:56116) 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 -- 172.21.15.96:0/1013892390 learned_addr learned my addr 172.21.15.96:0/1013892390 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d982850 0x560a5d984740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 0x560a5d97c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 -- 172.21.15.96:0/1013892390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d982850 msgr2=0x560a5d984740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d982850 0x560a5d984740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 -- 172.21.15.96:0/1013892390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 msgr2=0x560a5d97c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 0x560a5d97c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 -- 172.21.15.96:0/1013892390 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a5d6c57d0 con 0x560a5d983240 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 0x560a5d97c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffbb409b6f0 tx=0x7ffbb40a3fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a8c77388da8054 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc0ff9640 1 -- 172.21.15.96:0/1013892390 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40b4040 con 0x560a5d983240 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc0ff9640 1 -- 172.21.15.96:0/1013892390 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb40860b0 con 0x560a5d983240 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.528+0000 7ffbc0ff9640 1 -- 172.21.15.96:0/1013892390 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40a7070 con 0x560a5d983240 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- 172.21.15.96:0/1013892390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 msgr2=0x560a5d984c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffbb409b6f0 tx=0x7ffbb40a3fd0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- 172.21.15.96:0/1013892390 shutdown_connections 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d9883b0 0x560a5d97c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d983240 0x560a5d984c80 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- 172.21.15.96:0/1013892390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d982850 0x560a5d984740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- 172.21.15.96:0/1013892390 >> 172.21.15.96:0/1013892390 conn(0x560a5d97ddc0 msgr2=0x560a5d97e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- 172.21.15.96:0/1013892390 shutdown_connections 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- 172.21.15.96:0/1013892390 wait complete. 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- start start 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 Processor -- start 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d9f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d9fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a8100 con 0x560a5d9883b0 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a7a30 con 0x560a5d982850 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1a4c10 con 0x560a5d983240 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbd438e740 10 client.0 _ll_register_callbacks cb 0x7ffbb400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbcb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbcb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56126/0 (socket says 172.21.15.96:56126) 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbcb7fe640 1 -- 172.21.15.96:0/3758582400 learned_addr learned my addr 172.21.15.96:0/3758582400 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d9f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.529+0000 7ffbc3fff640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d9fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 -- 172.21.15.96:0/3758582400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 msgr2=0x560a5d9fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d9fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 -- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 msgr2=0x560a5d9f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d9f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 -- 172.21.15.96:0/3758582400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a5da01b30 con 0x560a5d9883b0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d9f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbcb7fe640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034720 tx=0x7ffbc4009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9b048776241b85 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc403b040 con 0x560a5d9883b0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ffbc4034bb0 con 0x560a5d9883b0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb40860b0 con 0x560a5d9883b0 2024-03-18T15:47:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbc4052030 con 0x560a5d9883b0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 10 client.17214 fetch_fsmap learned FSMap version 40 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a5da02ea0 con 0x560a5d9883b0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb40860b0 con 0x560a5d9883b0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 40) ==== 8374+0+0 (secure 0 0 0) 0x7ffbc404c470 con 0x560a5d9883b0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 10 client.17214 fetch_fsmap finished waiting for FSMap version 40 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 10 client.17214 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.530+0000 7ffbd438e740 20 client.17214 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5da05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 4 client.17214 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a5d8eed20 con 0x560a5d998a80 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5da05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbb27fc640 10 client.17214 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbcaffd640 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5da05940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7182559a81b994 server_cookie=bf962b53b99753fe in_seq=0 out_seq=0 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbb27fc640 1 -- 172.21.15.96:0/3758582400 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a5d8eed20 con 0x560a5d998a80 2024-03-18T15:47:09.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbb27fc640 10 client.17214 handle_command_reply: tid=0 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 client.17214 shutdown 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 msgr2=0x560a5da01540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4034720 tx=0x7ffbc4009960 comp rx=0 tx=0).stop 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 shutdown_connections 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x560a5d998a80 0x560a5da05940 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560a5d9883b0 0x560a5da01540 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560a5d983240 0x560a5d9fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbb27fc640 0 client.17214 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 --2- 172.21.15.96:0/3758582400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a5d982850 0x560a5d9f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 >> 172.21.15.96:0/3758582400 conn(0x560a5d97ddc0 msgr2=0x560a5d98cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.531+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 shutdown_connections 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd438e740 1 -- 172.21.15.96:0/3758582400 wait complete. 2024-03-18T15:47:09.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd438e740 20 client.17214 trim_cache size 0 max 16384 2024-03-18T15:47:09.541 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:09.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 msgr2=0x7ffbb80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 0x7ffbb80a4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc101730 tx=0x7ffbbc005f90 comp rx=0 tx=0).stop 2024-03-18T15:47:09.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 msgr2=0x7ffbcc195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400e9f0 tx=0x7ffbb400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 shutdown_connections 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ffbb80a2880 0x7ffbb80a4d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbcc102290 0x7ffbcc19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbcc1018d0 0x7ffbcc195df0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 --2- 172.21.15.96:0/4056596827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffbcc1006e0 0x7ffbcc1958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 >> 172.21.15.96:0/4056596827 conn(0x7ffbcc0fbc70 msgr2=0x7ffbcc104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 shutdown_connections 2024-03-18T15:47:09.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.532+0000 7ffbd16c6640 1 -- 172.21.15.96:0/4056596827 wait complete. 2024-03-18T15:47:09.544 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:47:09.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 -- 172.21.15.96:0/1558299377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f18100700 msgr2=0x7f4f18100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 --2- 172.21.15.96:0/1558299377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f18100700 0x7f4f18100b20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009900 tx=0x7f4f0802bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:09.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 -- 172.21.15.96:0/1558299377 shutdown_connections 2024-03-18T15:47:09.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 --2- 172.21.15.96:0/1558299377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f181022b0 0x7f4f18108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 --2- 172.21.15.96:0/1558299377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 --2- 172.21.15.96:0/1558299377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f18100700 0x7f4f18100b20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 -- 172.21.15.96:0/1558299377 >> 172.21.15.96:0/1558299377 conn(0x7f4f180fbc90 msgr2=0x7f4f180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 -- 172.21.15.96:0/1558299377 shutdown_connections 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.651+0000 7f4f1c932640 1 -- 172.21.15.96:0/1558299377 wait complete. 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 Processor -- start 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 -- start start 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f18100700 0x7f4f18196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c380 con 0x7f4f181022b0 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c200 con 0x7f4f181018f0 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1c932640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1810c500 con 0x7f4f18100700 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56144/0 (socket says 172.21.15.96:56144) 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 -- 172.21.15.96:0/2268554361 learned_addr learned my addr 172.21.15.96:0/2268554361 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f1659c640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f18100700 0x7f4f18196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 -- 172.21.15.96:0/2268554361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f18100700 msgr2=0x7f4f18196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f18100700 0x7f4f18196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 -- 172.21.15.96:0/2268554361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 msgr2=0x7f4f18196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f16d9d640 1 -- 172.21.15.96:0/2268554361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1819e630 con 0x7f4f181022b0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.652+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f16d9d640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000eb10 tx=0x7f4f1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f0f7fe640 1 -- 172.21.15.96:0/2268554361 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10004b20 con 0x7f4f181022b0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1819e920 con 0x7f4f181022b0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f0f7fe640 1 -- 172.21.15.96:0/2268554361 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f10004cc0 con 0x7f4f181022b0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f0f7fe640 1 -- 172.21.15.96:0/2268554361 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10010760 con 0x7f4f181022b0 2024-03-18T15:47:09.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.653+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1819eec0 con 0x7f4f181022b0 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.655+0000 7f4f0f7fe640 1 -- 172.21.15.96:0/2268554361 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f4f10005020 con 0x7f4f181022b0 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.655+0000 7f4f1f5f4740 1 Processor -- start 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 -- start start 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa782fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa78ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563daa6e7490 con 0x563daa782b60 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563daa68fc90 con 0x563daa78cb30 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563daa6e3530 con 0x563daa783ed0 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0f7fe640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 0x7f4f040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa782fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:41306/0 (socket says 172.21.15.96:41306) 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa782fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56146/0 (socket says 172.21.15.96:56146) 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3939811328 learned_addr learned my addr 172.21.15.96:0/3939811328 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f16d9d640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 0x7f4f040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa78ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0f7fe640 1 -- 172.21.15.96:0/2268554361 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4f100cf170 con 0x7f4f181022b0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3939811328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 msgr2=0x563daa78ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa78ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3939811328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 msgr2=0x563daa782fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa782fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.656+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3939811328 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563daa74a500 con 0x563daa783ed0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.657+0000 7f4f16d9d640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 0x7f4f040a4cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f100984b0 tx=0x7f4f100668a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.657+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011be0 tx=0x7f4f00009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1561e673477380c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.663+0000 7f4f0cff9640 1 -- 172.21.15.96:0/3939811328 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000fc50 con 0x563daa783ed0 2024-03-18T15:47:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.663+0000 7f4f0cff9640 1 -- 172.21.15.96:0/3939811328 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0000fdf0 con 0x563daa783ed0 2024-03-18T15:47:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.663+0000 7f4f0cff9640 1 -- 172.21.15.96:0/3939811328 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000e6a0 con 0x563daa783ed0 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3939811328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 msgr2=0x563daa782620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011be0 tx=0x7f4f00009ba0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3939811328 shutdown_connections 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa78ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa782fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3939811328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa782620 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3939811328 >> 172.21.15.96:0/3939811328 conn(0x563daa77ddc0 msgr2=0x563daa77e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3939811328 shutdown_connections 2024-03-18T15:47:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3939811328 wait complete. 2024-03-18T15:47:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.664+0000 7f4f1f5f4740 1 -- start start 2024-03-18T15:47:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 Processor -- start 2024-03-18T15:47:09.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa782b60 0x563daa7911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa78cb30 0x563daa798740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563daa78f760 con 0x563daa783ed0 2024-03-18T15:47:09.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563daa6a3230 con 0x563daa782b60 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563daa78f8e0 con 0x563daa78cb30 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1f5f4740 10 client.0 _ll_register_callbacks cb 0x563daa6df010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56150/0 (socket says 172.21.15.96:56150) 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 -- 172.21.15.96:0/1171297698 learned_addr learned my addr 172.21.15.96:0/1171297698 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa782b60 0x563daa7911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa78cb30 0x563daa798740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 -- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa78cb30 msgr2=0x563daa798740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa78cb30 0x563daa798740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 -- 172.21.15.96:0/1171297698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa782b60 msgr2=0x563daa7911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa782b60 0x563daa7911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.665+0000 7f4f1659c640 1 -- 172.21.15.96:0/1171297698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563daa6b2100 con 0x563daa783ed0 2024-03-18T15:47:09.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.666+0000 7f4f1659c640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009900 tx=0x7f4f08004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8e8a5531fe1c29 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.666+0000 7f4efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.666+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08005d50 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.666+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0000fdf0 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.666+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4f08005230 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4f08005430 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 10 client.17226 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563daa3fb3f0 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0000fdf0 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f4f0804f070 con 0x563daa783ed0 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 10 client.17226 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 10 client.17226 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 20 client.17226 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa79cfa0 0x563daa79f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 4 client.17226 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563daa6f49d0 con 0x563daa79cfa0 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.667+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa79cfa0 0x563daa79f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.668+0000 7f4efdffb640 10 client.17226 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.668+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa79cfa0 0x563daa79f440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a5703b7c40e45cbb server_cookie=4aa48acbc697947c in_seq=0 out_seq=0 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4efdffb640 1 -- 172.21.15.96:0/1171297698 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563daa6f49d0 con 0x563daa79cfa0 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4efdffb640 10 client.17226 handle_command_reply: tid=0 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 client.17226 shutdown 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 msgr2=0x563daa791700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009900 tx=0x7f4f08004270 comp rx=0 tx=0).stop 2024-03-18T15:47:09.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 shutdown_connections 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa79cfa0 0x563daa79f440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa78cb30 0x563daa798740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa791700 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/1171297698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa782b60 0x563daa7911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4efdffb640 0 client.17226 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 >> 172.21.15.96:0/1171297698 conn(0x563daa77ddc0 msgr2=0x563daa77cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 shutdown_connections 2024-03-18T15:47:09.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/1171297698 wait complete. 2024-03-18T15:47:09.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.669+0000 7f4f1f5f4740 20 client.17226 trim_cache size 0 max 16384 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 Processor -- start 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 -- start start 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa798700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa7a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a6ba0 con 0x563daa782b60 2024-03-18T15:47:09.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a8770 con 0x563daa78cb30 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a72a0 con 0x563daa783ed0 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56164/0 (socket says 172.21.15.96:56164) 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3566372359 learned_addr learned my addr 172.21.15.96:0/3566372359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.682+0000 7f4f1659c640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa798700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa7a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3566372359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 msgr2=0x563daa798700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa798700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3566372359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 msgr2=0x563daa7a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa7a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 -- 172.21.15.96:0/3566372359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563daa78f290 con 0x563daa782b60 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f1659c640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa798700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:09.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009730 tx=0x7f4f00009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141f8af1f53b287 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4efffff640 1 -- 172.21.15.96:0/3566372359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000fd50 con 0x563daa782b60 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4efffff640 1 -- 172.21.15.96:0/3566372359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f00005510 con 0x563daa782b60 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4efffff640 1 -- 172.21.15.96:0/3566372359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00030310 con 0x563daa782b60 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3566372359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 msgr2=0x563daa781730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.683+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009730 tx=0x7f4f00009f40 comp rx=0 tx=0).stop 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3566372359 shutdown_connections 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa7a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa783ed0 0x563daa798700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/3566372359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa782b60 0x563daa781730 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3566372359 >> 172.21.15.96:0/3566372359 conn(0x563daa77ddc0 msgr2=0x563daa781020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3566372359 shutdown_connections 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/3566372359 wait complete. 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- start start 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 Processor -- start 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa782b60 0x563daa795d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa797250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a80a0 con 0x563daa783ed0 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a79a0 con 0x563daa78cb30 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181a4b80 con 0x563daa782b60 2024-03-18T15:47:09.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1f5f4740 10 client.0 _ll_register_callbacks cb 0x7f4f08009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56180/0 (socket says 172.21.15.96:56180) 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 -- 172.21.15.96:0/119348294 learned_addr learned my addr 172.21.15.96:0/119348294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa782b60 0x563daa795d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa797250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 -- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa782b60 msgr2=0x563daa795d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa782b60 0x563daa795d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 -- 172.21.15.96:0/119348294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 msgr2=0x563daa797250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa797250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f1659c640 1 -- 172.21.15.96:0/119348294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563daa4c57d0 con 0x563daa783ed0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.684+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa797250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1659c640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4f0802bc70 tx=0x7f4f08005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d2100ae2f15ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08005d50 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f4f080518a0 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563daa6eed20 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4f080035f0 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 10 client.17232 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563daa7989a0 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563daa6eed20 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f4f08059030 con 0x563daa783ed0 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 10 client.17232 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 10 client.17232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 20 client.17232 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa803a40 0x563daa805e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 4 client.17232 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:09.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563daa66c6a0 con 0x563daa803a40 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.685+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa803a40 0x563daa805e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4efdffb640 10 client.17232 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f0ffff640 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa803a40 0x563daa805e60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc0afcd17a8f1722 server_cookie=15e14a981c4bb008 in_seq=0 out_seq=0 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4efdffb640 1 -- 172.21.15.96:0/119348294 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563daa66c6a0 con 0x563daa803a40 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4efdffb640 10 client.17232 handle_command_reply: tid=0 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 client.17232 shutdown 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 msgr2=0x563daa796240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4f0802bc70 tx=0x7f4f08005800 comp rx=0 tx=0).stop 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 shutdown_connections 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x563daa803a40 0x563daa805e60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563daa78cb30 0x563daa797250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4efdffb640 0 client.17232 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:09.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563daa783ed0 0x563daa796240 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 --2- 172.21.15.96:0/119348294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563daa782b60 0x563daa795d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 >> 172.21.15.96:0/119348294 conn(0x563daa77ddc0 msgr2=0x563daa7818b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 shutdown_connections 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 1 -- 172.21.15.96:0/119348294 wait complete. 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.686+0000 7f4f1f5f4740 20 client.17232 trim_cache size 0 max 16384 2024-03-18T15:47:09.696 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 msgr2=0x7f4f040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 0x7f4f040a4cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f100984b0 tx=0x7f4f100668a0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 msgr2=0x7f4f1819dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000eb10 tx=0x7f4f1000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 shutdown_connections 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f4f040a2800 0x7f4f040a4cc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f181022b0 0x7f4f1819dfc0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f181018f0 0x7f4f18196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 --2- 172.21.15.96:0/2268554361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f18100700 0x7f4f18196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 >> 172.21.15.96:0/2268554361 conn(0x7f4f180fbc90 msgr2=0x7f4f18104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 shutdown_connections 2024-03-18T15:47:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.687+0000 7f4f1c932640 1 -- 172.21.15.96:0/2268554361 wait complete. 2024-03-18T15:47:09.699 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 -- 172.21.15.96:0/2720625087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580107520 msgr2=0x7f7580109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 --2- 172.21.15.96:0/2720625087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580107520 0x7f7580109910 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7578009ca0 tx=0x7f757802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 -- 172.21.15.96:0/2720625087 shutdown_connections 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 --2- 172.21.15.96:0/2720625087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580107520 0x7f7580109910 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 --2- 172.21.15.96:0/2720625087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7580104bf0 0x7f7580106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 --2- 172.21.15.96:0/2720625087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75801022c0 0x7f75801046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 -- 172.21.15.96:0/2720625087 >> 172.21.15.96:0/2720625087 conn(0x7f75800fbc70 msgr2=0x7f75800fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 -- 172.21.15.96:0/2720625087 shutdown_connections 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.803+0000 7f75869f4640 1 -- 172.21.15.96:0/2720625087 wait complete. 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 Processor -- start 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 -- start start 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75801022c0 0x7f7580197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580107520 0x7f758019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f758010d480 con 0x7f7580104bf0 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f758010d300 con 0x7f7580107520 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f75869f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f758010d600 con 0x7f75801022c0 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56206/0 (socket says 172.21.15.96:56206) 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f757ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75801022c0 0x7f7580197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f757f7fe640 1 -- 172.21.15.96:0/2363188670 learned_addr learned my addr 172.21.15.96:0/2363188670 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.804+0000 7f7584f91640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580107520 0x7f758019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 -- 172.21.15.96:0/2363188670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75801022c0 msgr2=0x7f7580197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75801022c0 0x7f7580197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 -- 172.21.15.96:0/2363188670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580107520 msgr2=0x7f758019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580107520 0x7f758019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 -- 172.21.15.96:0/2363188670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758019f720 con 0x7f7580104bf0 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757f7fe640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f757400e9f0 tx=0x7f757400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757d7fa640 1 -- 172.21.15.96:0/2363188670 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757400ccc0 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757d7fa640 1 -- 172.21.15.96:0/2363188670 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7574005500 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758019fa10 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f757d7fa640 1 -- 172.21.15.96:0/2363188670 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574010690 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.805+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758019ff10 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.807+0000 7f757d7fa640 1 -- 172.21.15.96:0/2363188670 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f75740046c0 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757d7fa640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 0x7f75680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 Processor -- start 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 -- start start 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 0x7f75680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757d7fa640 1 -- 172.21.15.96:0/2363188670 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f75740cedd0 con 0x7f7580104bf0 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 0x55dd1abe7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd1ab47490 con 0x55dd1abe2850 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd1aaefc90 con 0x55dd1abe83b0 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f75895e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd1ab43530 con 0x55dd1abe3240 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56220/0 (socket says 172.21.15.96:56220) 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 -- 172.21.15.96:0/28261492 learned_addr learned my addr 172.21.15.96:0/28261492 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 0x55dd1abe7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f7584f91640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 0x7f75680a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7570013520 tx=0x7f7570013920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 -- 172.21.15.96:0/28261492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 msgr2=0x55dd1abea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 -- 172.21.15.96:0/28261492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 msgr2=0x55dd1abe2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757f7fe640 1 -- 172.21.15.96:0/28261492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1abaa500 con 0x55dd1abe3240 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.808+0000 7f757ffff640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f757f7fe640 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 0x55dd1abe7d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75740ccc10 tx=0x7f757409b5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fcbe1ce057d5a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75667fc640 1 -- 172.21.15.96:0/28261492 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b4070 con 0x55dd1abe3240 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75667fc640 1 -- 172.21.15.96:0/28261492 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757409b740 con 0x55dd1abe3240 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75667fc640 1 -- 172.21.15.96:0/28261492 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740a3a50 con 0x55dd1abe3240 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 -- 172.21.15.96:0/28261492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 msgr2=0x55dd1abe7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 0x55dd1abe7d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f75740ccc10 tx=0x7f757409b5d0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 -- 172.21.15.96:0/28261492 shutdown_connections 2024-03-18T15:47:09.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe3240 0x55dd1abe7d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 --2- 172.21.15.96:0/28261492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe2850 0x55dd1abe2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 -- 172.21.15.96:0/28261492 >> 172.21.15.96:0/28261492 conn(0x55dd1abdddc0 msgr2=0x55dd1abde1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.809+0000 7f75895e3740 1 -- 172.21.15.96:0/28261492 shutdown_connections 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 -- 172.21.15.96:0/28261492 wait complete. 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 -- start start 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 Processor -- start 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe2850 0x55dd1abecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe83b0 0x55dd1abf41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd1abeb360 con 0x55dd1abe3240 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dd1ab03230 con 0x55dd1abe2850 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd1abeb4e0 con 0x55dd1abe83b0 2024-03-18T15:47:09.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f75895e3740 10 client.0 _ll_register_callbacks cb 0x55dd1ab3f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56224/0 (socket says 172.21.15.96:56224) 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f757f7fe640 1 -- 172.21.15.96:0/27416997 learned_addr learned my addr 172.21.15.96:0/27416997 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f757ffff640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe2850 0x55dd1abecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.810+0000 7f7584f91640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe83b0 0x55dd1abf41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 -- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe83b0 msgr2=0x55dd1abf41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe83b0 0x55dd1abf41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 -- 172.21.15.96:0/27416997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe2850 msgr2=0x55dd1abecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe2850 0x55dd1abecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 -- 172.21.15.96:0/27416997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1ab12100 con 0x55dd1abe3240 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f757f7fe640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f757400cf10 tx=0x7f757400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae767a130eb9c2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b4040 con 0x55dd1abe3240 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f75740a3c50 con 0x55dd1abe3240 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757409b740 con 0x55dd1abe3240 2024-03-18T15:47:09.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f757409f890 con 0x55dd1abe3240 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 10 client.17244 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dd1a85b3f0 con 0x55dd1abe3240 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757409b740 con 0x55dd1abe3240 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f757409a800 con 0x55dd1abe3240 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 10 client.17244 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 10 client.17244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 20 client.17244 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1abf86e0 0x55dd1abfaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 4 client.17244 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.811+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd1ab549d0 con 0x55dd1abf86e0 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.812+0000 7f757ffff640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1abf86e0 0x55dd1abfaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.812+0000 7f755f7fe640 10 client.17244 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.812+0000 7f757ffff640 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1abf86e0 0x55dd1abfaad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7951123e8dc70640 server_cookie=30f67598456bbd1b in_seq=0 out_seq=0 2024-03-18T15:47:09.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.814+0000 7f755f7fe640 1 -- 172.21.15.96:0/27416997 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55dd1ab549d0 con 0x55dd1abf86e0 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.814+0000 7f755f7fe640 10 client.17244 handle_command_reply: tid=0 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 client.17244 shutdown 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 msgr2=0x55dd1abed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f757400cf10 tx=0x7f757400cf60 comp rx=0 tx=0).stop 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 shutdown_connections 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1abf86e0 0x55dd1abfaad0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe83b0 0x55dd1abf41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abed1d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 --2- 172.21.15.96:0/27416997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe2850 0x55dd1abecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f755f7fe640 0 client.17244 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 >> 172.21.15.96:0/27416997 conn(0x55dd1abdddc0 msgr2=0x55dd1abdcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 shutdown_connections 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 1 -- 172.21.15.96:0/27416997 wait complete. 2024-03-18T15:47:09.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.815+0000 7f75895e3740 20 client.17244 trim_cache size 0 max 16384 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 Processor -- start 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 -- start start 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abdc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 0x55dd1abf40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a8e00 con 0x55dd1abe83b0 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75801aa9d0 con 0x55dd1abe3240 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f75895e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a9500 con 0x55dd1abe2850 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f7584f91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f757ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abdc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f7584f91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56230/0 (socket says 172.21.15.96:56230) 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f7584f91640 1 -- 172.21.15.96:0/158438450 learned_addr learned my addr 172.21.15.96:0/158438450 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.828+0000 7f757f7fe640 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 0x55dd1abf40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 -- 172.21.15.96:0/158438450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 msgr2=0x55dd1abdc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abdc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 -- 172.21.15.96:0/158438450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 msgr2=0x55dd1abf40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 0x55dd1abf40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 -- 172.21.15.96:0/158438450 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd1a9257d0 con 0x55dd1abe83b0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f757f7fe640 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 0x55dd1abf40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7584f91640 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f75780324c0 tx=0x7f7578032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f93b9d538ce6772 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7565ffb640 1 -- 172.21.15.96:0/158438450 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578039040 con 0x55dd1abe83b0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7565ffb640 1 -- 172.21.15.96:0/158438450 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7578032c90 con 0x55dd1abe83b0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f7565ffb640 1 -- 172.21.15.96:0/158438450 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578004040 con 0x55dd1abe83b0 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 -- 172.21.15.96:0/158438450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 msgr2=0x55dd1abdcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f75780324c0 tx=0x7f7578032590 comp rx=0 tx=0).stop 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 -- 172.21.15.96:0/158438450 shutdown_connections 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe83b0 0x55dd1abdcc90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe3240 0x55dd1abf40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.829+0000 7f75895e3740 1 --2- 172.21.15.96:0/158438450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abdc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- 172.21.15.96:0/158438450 >> 172.21.15.96:0/158438450 conn(0x55dd1abdddc0 msgr2=0x55dd1abe46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- 172.21.15.96:0/158438450 shutdown_connections 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- 172.21.15.96:0/158438450 wait complete. 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- start start 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 Processor -- start 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abf1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abf29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75801aa300 con 0x55dd1abe3240 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a9c00 con 0x55dd1abe83b0 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75801a6de0 con 0x55dd1abe2850 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f75895e3740 10 client.0 _ll_register_callbacks cb 0x7f7570005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f757f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56242/0 (socket says 172.21.15.96:56242) 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f757f7fe640 1 -- 172.21.15.96:0/3544349841 learned_addr learned my addr 172.21.15.96:0/3544349841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f757ffff640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abf1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.830+0000 7f7584f91640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abf29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 -- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 msgr2=0x55dd1abf1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abf1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 -- 172.21.15.96:0/3544349841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 msgr2=0x55dd1abf29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abf29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 -- 172.21.15.96:0/3544349841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd1abf2f90 con 0x55dd1abe3240 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f757f7fe640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f757400c6a0 tx=0x7f7574004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc43c5c45a09ead server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740056b0 con 0x55dd1abe3240 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f757409ad90 con 0x55dd1abe3240 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7578032c90 con 0x55dd1abe3240 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f757409b740 con 0x55dd1abe3240 2024-03-18T15:47:09.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 10 client.17250 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55dd1abf4290 con 0x55dd1abe3240 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7578032c90 con 0x55dd1abe3240 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f757409a600 con 0x55dd1abe3240 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 10 client.17250 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 20 client.17250 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1ac5edb0 0x55dd1ac611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 4 client.17250 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.831+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd1ab4ed20 con 0x55dd1ac5edb0 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.832+0000 7f757ffff640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1ac5edb0 0x55dd1ac611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.832+0000 7f755f7fe640 10 client.17250 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.832+0000 7f757ffff640 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1ac5edb0 0x55dd1ac611d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc0ae7206f6f3ef8 server_cookie=9da66bc63b442822 in_seq=0 out_seq=0 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f755f7fe640 1 -- 172.21.15.96:0/3544349841 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd1ab4ed20 con 0x55dd1ac5edb0 2024-03-18T15:47:09.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f755f7fe640 10 client.17250 handle_command_reply: tid=0 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 client.17250 shutdown 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 msgr2=0x55dd1abf1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f757400c6a0 tx=0x7f7574004aa0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 shutdown_connections 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55dd1ac5edb0 0x55dd1ac611d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd1abe83b0 0x55dd1abf29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd1abe3240 0x55dd1abf1990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 --2- 172.21.15.96:0/3544349841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd1abe2850 0x55dd1abf1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f755f7fe640 0 client.17250 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 >> 172.21.15.96:0/3544349841 conn(0x55dd1abdddc0 msgr2=0x55dd1abe1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 shutdown_connections 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 1 -- 172.21.15.96:0/3544349841 wait complete. 2024-03-18T15:47:09.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.836+0000 7f75895e3740 20 client.17250 trim_cache size 0 max 16384 2024-03-18T15:47:09.848 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:09.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 msgr2=0x7f75680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 0x7f75680a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7570013520 tx=0x7f7570013920 comp rx=0 tx=0).stop 2024-03-18T15:47:09.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 msgr2=0x7f7580198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f757400e9f0 tx=0x7f757400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 shutdown_connections 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f75680a2800 0x7f75680a4cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580107520 0x7f758019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7580104bf0 0x7f7580198040 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 --2- 172.21.15.96:0/2363188670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75801022c0 0x7f7580197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 >> 172.21.15.96:0/2363188670 conn(0x7f75800fbc70 msgr2=0x7f7580102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 shutdown_connections 2024-03-18T15:47:09.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.837+0000 7f75869f4640 1 -- 172.21.15.96:0/2363188670 wait complete. 2024-03-18T15:47:09.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:47:09.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:09 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1904483482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:09.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:09 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1904483482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:09.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 -- 172.21.15.96:0/2461415362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8568108ff0 msgr2=0x7f8568109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 --2- 172.21.15.96:0/2461415362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8568108ff0 0x7f8568109470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f855c009900 tx=0x7f855c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 -- 172.21.15.96:0/2461415362 shutdown_connections 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 --2- 172.21.15.96:0/2461415362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f85681099b0 0x7f8568110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 --2- 172.21.15.96:0/2461415362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8568108ff0 0x7f8568109470 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 --2- 172.21.15.96:0/2461415362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568107e00 0x7f8568108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 -- 172.21.15.96:0/2461415362 >> 172.21.15.96:0/2461415362 conn(0x7f8568076da0 msgr2=0x7f85680791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 -- 172.21.15.96:0/2461415362 shutdown_connections 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.958+0000 7f856c861640 1 -- 172.21.15.96:0/2461415362 wait complete. 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 Processor -- start 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 -- start start 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 0x7f856819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568108ff0 0x7f856819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8568113a80 con 0x7f85681099b0 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8568113900 con 0x7f8568107e00 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856c861640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8568113c00 con 0x7f8568108ff0 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 0x7f856819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56262/0 (socket says 172.21.15.96:56262) 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 -- 172.21.15.96:0/2322159627 learned_addr learned my addr 172.21.15.96:0/2322159627 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8565d9b640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568108ff0 0x7f856819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 -- 172.21.15.96:0/2322159627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568108ff0 msgr2=0x7f856819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568108ff0 0x7f856819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 -- 172.21.15.96:0/2322159627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 msgr2=0x7f856819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 0x7f856819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 -- 172.21.15.96:0/2322159627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85681a5d20 con 0x7f85681099b0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f856659c640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 0x7f856819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.959+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f856000eb10 tx=0x7f856000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.960+0000 7f85577fe640 1 -- 172.21.15.96:0/2322159627 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560004b20 con 0x7f85681099b0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.960+0000 7f85577fe640 1 -- 172.21.15.96:0/2322159627 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8560004cc0 con 0x7f85681099b0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.960+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85681a5fb0 con 0x7f85681099b0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.960+0000 7f85577fe640 1 -- 172.21.15.96:0/2322159627 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560010760 con 0x7f85681099b0 2024-03-18T15:47:09.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.960+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a6480 con 0x7f85681099b0 2024-03-18T15:47:09.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f85577fe640 1 -- 172.21.15.96:0/2322159627 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f8560005020 con 0x7f85681099b0 2024-03-18T15:47:09.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f856f453740 1 Processor -- start 2024-03-18T15:47:09.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f856f453740 1 -- start start 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f85577fe640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 0x7f85500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f856f453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062018bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.962+0000 7f856f453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f856f453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f856f453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5606200f0490 con 0x56062018c240 2024-03-18T15:47:09.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f856f453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560620098c90 con 0x5606201913b0 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f856f453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606200ec530 con 0x56062018b850 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 0x7f85500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f85577fe640 1 -- 172.21.15.96:0/2322159627 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f85600cf000 con 0x7f85681099b0 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56268/0 (socket says 172.21.15.96:56268) 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 -- 172.21.15.96:0/968855547 learned_addr learned my addr 172.21.15.96:0/968855547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55336/0 (socket says 172.21.15.96:55336) 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f856659c640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062018bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 -- 172.21.15.96:0/968855547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 msgr2=0x56062018bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062018bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 -- 172.21.15.96:0/968855547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 msgr2=0x560620190d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 -- 172.21.15.96:0/968855547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560620153500 con 0x5606201913b0 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8565d9b640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 0x7f85500a4cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f855c02cfd0 tx=0x7f855c0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8566d9d640 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85600ccb20 tx=0x7f856009b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0e77d4f984763a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.963+0000 7f8554ff9640 1 -- 172.21.15.96:0/968855547 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856009c070 con 0x5606201913b0 2024-03-18T15:47:09.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f8554ff9640 1 -- 172.21.15.96:0/968855547 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856009b970 con 0x5606201913b0 2024-03-18T15:47:09.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f8554ff9640 1 -- 172.21.15.96:0/968855547 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856009bbd0 con 0x5606201913b0 2024-03-18T15:47:09.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 -- 172.21.15.96:0/968855547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 msgr2=0x560620193840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f85600ccb20 tx=0x7f856009b740 comp rx=0 tx=0).stop 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 -- 172.21.15.96:0/968855547 shutdown_connections 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x560620193840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018c240 0x560620190d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 --2- 172.21.15.96:0/968855547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062018bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 -- 172.21.15.96:0/968855547 >> 172.21.15.96:0/968855547 conn(0x560620186dc0 msgr2=0x5606201871c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 -- 172.21.15.96:0/968855547 shutdown_connections 2024-03-18T15:47:09.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.964+0000 7f856f453740 1 -- 172.21.15.96:0/968855547 wait complete. 2024-03-18T15:47:09.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 -- start start 2024-03-18T15:47:09.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 Processor -- start 2024-03-18T15:47:09.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 0x560620196270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x56062019d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560620194360 con 0x56062018b850 2024-03-18T15:47:09.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5606200ac230 con 0x5606201913b0 2024-03-18T15:47:09.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606201944e0 con 0x56062018c240 2024-03-18T15:47:09.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856f453740 10 client.0 _ll_register_callbacks cb 0x5606200e8010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56280/0 (socket says 172.21.15.96:56280) 2024-03-18T15:47:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 -- 172.21.15.96:0/3139927984 learned_addr learned my addr 172.21.15.96:0/3139927984 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f8566d9d640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x56062019d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f8565d9b640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 0x560620196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 -- 172.21.15.96:0/3139927984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 msgr2=0x560620196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 0x560620196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 -- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 msgr2=0x56062019d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x56062019d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 -- 172.21.15.96:0/3139927984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606200bb100 con 0x56062018b850 2024-03-18T15:47:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f8565d9b640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 0x560620196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f856659c640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8558012a20 tx=0x7f8558012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efff33ad773a80f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.965+0000 7f854dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800faa0 con 0x56062018b850 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f855800fc40 con 0x56062018b850 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f856009b970 con 0x56062018b850 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8558022030 con 0x56062018b850 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 10 client.17262 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56061fe043f0 con 0x56062018b850 2024-03-18T15:47:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856009b970 con 0x56062018b850 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f8558003c80 con 0x56062018b850 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 10 client.17262 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 10 client.17262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 20 client.17262 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x5606201a1ac0 0x5606201a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 4 client.17262 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606200fd9d0 con 0x5606201a1ac0 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.966+0000 7f8565d9b640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x5606201a1ac0 0x5606201a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.967+0000 7f854dffb640 10 client.17262 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.967+0000 7f8565d9b640 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x5606201a1ac0 0x5606201a3f40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=243105a78c14da2 server_cookie=b546df3faabccdfd in_seq=0 out_seq=0 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.967+0000 7f854dffb640 1 -- 172.21.15.96:0/3139927984 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5606200fd9d0 con 0x5606201a1ac0 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.967+0000 7f854dffb640 10 client.17262 handle_command_reply: tid=0 2024-03-18T15:47:09.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.967+0000 7f856f453740 1 client.17262 shutdown 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 msgr2=0x560620195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8558012a20 tx=0x7f8558012ee0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 shutdown_connections 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x5606201a1ac0 0x5606201a3f40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606201913b0 0x56062019d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f854dffb640 0 client.17262 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018c240 0x560620196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 --2- 172.21.15.96:0/3139927984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x560620195d30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 >> 172.21.15.96:0/3139927984 conn(0x560620186dc0 msgr2=0x560620185fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 shutdown_connections 2024-03-18T15:47:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 1 -- 172.21.15.96:0/3139927984 wait complete. 2024-03-18T15:47:09.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.968+0000 7f856f453740 20 client.17262 trim_cache size 0 max 16384 2024-03-18T15:47:09.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.981+0000 7f856f453740 1 Processor -- start 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 -- start start 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606201913b0 0x560620185750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85681af330 con 0x56062018b850 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85681b0f00 con 0x56062018c240 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856f453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85681afa30 con 0x5606201913b0 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56296/0 (socket says 172.21.15.96:56296) 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 -- 172.21.15.96:0/1821930007 learned_addr learned my addr 172.21.15.96:0/1821930007 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f8566d9d640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606201913b0 0x560620185750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f8565d9b640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 -- 172.21.15.96:0/1821930007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606201913b0 msgr2=0x560620185750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:09.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606201913b0 0x560620185750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 -- 172.21.15.96:0/1821930007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 msgr2=0x56062019d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 -- 172.21.15.96:0/1821930007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56061fece7d0 con 0x56062018b850 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f8565d9b640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f856659c640 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8558027930 tx=0x7f8558013fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c127ea31427aace server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f854ffff640 1 -- 172.21.15.96:0/1821930007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558000e60 con 0x56062018b850 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f854ffff640 1 -- 172.21.15.96:0/1821930007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855800faa0 con 0x56062018b850 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.982+0000 7f854ffff640 1 -- 172.21.15.96:0/1821930007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580055d0 con 0x56062018b850 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- 172.21.15.96:0/1821930007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 msgr2=0x5606201961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8558027930 tx=0x7f8558013fc0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- 172.21.15.96:0/1821930007 shutdown_connections 2024-03-18T15:47:10.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606201913b0 0x560620185750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- 172.21.15.96:0/1821930007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56062018b850 0x5606201961d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- 172.21.15.96:0/1821930007 >> 172.21.15.96:0/1821930007 conn(0x560620186dc0 msgr2=0x560620195ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- 172.21.15.96:0/1821930007 shutdown_connections 2024-03-18T15:47:10.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- 172.21.15.96:0/1821930007 wait complete. 2024-03-18T15:47:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- start start 2024-03-18T15:47:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 Processor -- start 2024-03-18T15:47:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 0x56062019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.983+0000 7f856f453740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85681b0830 con 0x5606201913b0 2024-03-18T15:47:10.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f85681b0130 con 0x56062018c240 2024-03-18T15:47:10.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85681ad310 con 0x56062018b850 2024-03-18T15:47:10.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 10 client.0 _ll_register_callbacks cb 0x7f8558012600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 0x56062019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8565d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:41432/0 (socket says 172.21.15.96:41432) 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856659c640 1 -- 172.21.15.96:0/2958286794 learned_addr learned my addr 172.21.15.96:0/2958286794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 -- 172.21.15.96:0/2958286794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 msgr2=0x56062019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 -- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 msgr2=0x56062019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 -- 172.21.15.96:0/2958286794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56062019bfa0 con 0x5606201913b0 2024-03-18T15:47:10.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856659c640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:10.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8565d9b640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:10.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f8566d9d640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 0x56062019b9b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8560098340 tx=0x7f8560009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52223acb4235288 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f854dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:10.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600a4110 con 0x5606201913b0 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f85600a3c80 con 0x5606201913b0 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f855800faa0 con 0x5606201913b0 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f856009fac0 con 0x5606201913b0 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 10 client.17268 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56062019d2a0 con 0x5606201913b0 2024-03-18T15:47:10.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.984+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f855800faa0 con 0x5606201913b0 2024-03-18T15:47:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f85600860b0 con 0x5606201913b0 2024-03-18T15:47:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 10 client.17268 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 10 client.17268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:47:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 20 client.17268 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:10.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x560620207ed0 0x56062020a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 4 client.17268 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606200f7d20 con 0x560620207ed0 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856659c640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x560620207ed0 0x56062020a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f854dffb640 10 client.17268 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856659c640 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x560620207ed0 0x56062020a2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9455da38cf7ea32d server_cookie=f94e61c12cb2184c in_seq=0 out_seq=0 2024-03-18T15:47:10.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f854dffb640 1 -- 172.21.15.96:0/2958286794 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606200f7d20 con 0x560620207ed0 2024-03-18T15:47:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f854dffb640 10 client.17268 handle_command_reply: tid=0 2024-03-18T15:47:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.985+0000 7f856f453740 1 client.17268 shutdown 2024-03-18T15:47:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 msgr2=0x56062019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 0x56062019b9b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8560098340 tx=0x7f8560009200 comp rx=0 tx=0).stop 2024-03-18T15:47:10.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 shutdown_connections 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5606201913b0 0x56062019b9b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x560620207ed0 0x56062020a2c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56062018c240 0x56062019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f854dffb640 0 client.17268 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 --2- 172.21.15.96:0/2958286794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56062018b850 0x56062019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 >> 172.21.15.96:0/2958286794 conn(0x560620186dc0 msgr2=0x56062018a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 shutdown_connections 2024-03-18T15:47:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 1 -- 172.21.15.96:0/2958286794 wait complete. 2024-03-18T15:47:10.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856f453740 20 client.17268 trim_cache size 0 max 16384 2024-03-18T15:47:10.013 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:10.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 msgr2=0x7f85500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 0x7f85500a4cc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f855c02cfd0 tx=0x7f855c0095f0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 msgr2=0x7f85681a56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f856000eb10 tx=0x7f856000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 shutdown_connections 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f85500a2800 0x7f85500a4cc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85681099b0 0x7f85681a56b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8568108ff0 0x7f856819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 --2- 172.21.15.96:0/2322159627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8568107e00 0x7f856819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 >> 172.21.15.96:0/2322159627 conn(0x7f8568076da0 msgr2=0x7f856810c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 shutdown_connections 2024-03-18T15:47:10.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:09.986+0000 7f856c861640 1 -- 172.21.15.96:0/2322159627 wait complete. 2024-03-18T15:47:10.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:47:10.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:09 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1904483482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 -- 172.21.15.96:0/2635966606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 msgr2=0x7fb510100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 --2- 172.21.15.96:0/2635966606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510100b00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb500009ca0 tx=0x7fb50002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 -- 172.21.15.96:0/2635966606 shutdown_connections 2024-03-18T15:47:10.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 --2- 172.21.15.96:0/2635966606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb510108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 --2- 172.21.15.96:0/2635966606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 0x7fb510101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 --2- 172.21.15.96:0/2635966606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510100b00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 -- 172.21.15.96:0/2635966606 >> 172.21.15.96:0/2635966606 conn(0x7fb5100fbc70 msgr2=0x7fb5100fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 -- 172.21.15.96:0/2635966606 shutdown_connections 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.124+0000 7fb515a0b640 1 -- 172.21.15.96:0/2635966606 wait complete. 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 Processor -- start 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 -- start start 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 0x7fb5101980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb51019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c350 con 0x7fb5101006e0 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c1d0 con 0x7fb510102290 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb515a0b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb51010c4d0 con 0x7fb5101018d0 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb50effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb50effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56328/0 (socket says 172.21.15.96:56328) 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb50effd640 1 -- 172.21.15.96:0/1283858845 learned_addr learned my addr 172.21.15.96:0/1283858845 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb51019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.125+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 0x7fb5101980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 -- 172.21.15.96:0/1283858845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 msgr2=0x7fb5101980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 0x7fb5101980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 -- 172.21.15.96:0/1283858845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 msgr2=0x7fb51019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb51019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 -- 172.21.15.96:0/1283858845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51019f7b0 con 0x7fb5101006e0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb51019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb50effd640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb500000f80 tx=0x7fb500032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb514a09640 1 -- 172.21.15.96:0/1283858845 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500041020 con 0x7fb5101006e0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb514a09640 1 -- 172.21.15.96:0/1283858845 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb500034cd0 con 0x7fb5101006e0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51019fa40 con 0x7fb5101006e0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb514a09640 1 -- 172.21.15.96:0/1283858845 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500039040 con 0x7fb5101006e0 2024-03-18T15:47:10.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.126+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51019ff00 con 0x7fb5101006e0 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.128+0000 7fb514a09640 1 -- 172.21.15.96:0/1283858845 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb500044020 con 0x7fb5101006e0 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb514a09640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 0x7fb4f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 Processor -- start 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 -- start start 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 0x7fb4f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb514a09640 1 -- 172.21.15.96:0/1283858845 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb5000f07f0 con 0x7fb5101006e0 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640ec240 0x5644640f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564464050490 con 0x5644640f13b0 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564463ff8c90 con 0x5644640ec240 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb5185f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56446404c530 con 0x5644640eb850 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56332/0 (socket says 172.21.15.96:56332) 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2577007634 learned_addr learned my addr 172.21.15.96:0/2577007634 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640ec240 0x5644640f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 0x7fb4f80a4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb5101990a0 tx=0x7fb504011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2577007634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 msgr2=0x5644640ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2577007634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640ec240 msgr2=0x5644640f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640ec240 0x5644640f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2577007634 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644640b3500 con 0x5644640f13b0 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.129+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb508011540 tx=0x7fb508011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdb0e1d4f8c8276 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb4f5ffb640 1 -- 172.21.15.96:0/2577007634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50800f5c0 con 0x5644640f13b0 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb4f5ffb640 1 -- 172.21.15.96:0/2577007634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50800fc40 con 0x5644640f13b0 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb4f5ffb640 1 -- 172.21.15.96:0/2577007634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508019d20 con 0x5644640f13b0 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 -- 172.21.15.96:0/2577007634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 msgr2=0x5644640f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb508011540 tx=0x7fb508011a00 comp rx=0 tx=0).stop 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 -- 172.21.15.96:0/2577007634 shutdown_connections 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640f3840 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640ec240 0x5644640f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2577007634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 -- 172.21.15.96:0/2577007634 >> 172.21.15.96:0/2577007634 conn(0x5644640e6dc0 msgr2=0x5644640e71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 -- 172.21.15.96:0/2577007634 shutdown_connections 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.130+0000 7fb5185f9740 1 -- 172.21.15.96:0/2577007634 wait complete. 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 -- start start 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 Processor -- start 2024-03-18T15:47:10.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640eb850 0x5644640f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5644640f4360 con 0x5644640f13b0 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56446400c230 con 0x5644640eb850 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644640f44e0 con 0x5644640ec240 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb5185f9740 10 client.0 _ll_register_callbacks cb 0x564464048010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56342/0 (socket says 172.21.15.96:56342) 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2788277767 learned_addr learned my addr 172.21.15.96:0/2788277767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50effd640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640eb850 0x5644640f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 msgr2=0x5644640f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2788277767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640eb850 msgr2=0x5644640f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640eb850 0x5644640f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.131+0000 7fb50f7fe640 1 -- 172.21.15.96:0/2788277767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446401b100 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb50801b040 tx=0x7fb508009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbae45190dd6bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508019cb0 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb508004040 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb50800fc40 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb508019e50 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 10 client.17280 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x564463d643f0 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb50800fc40 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fb5080039f0 con 0x5644640f13b0 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 10 client.17280 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 20 client.17280 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564464101900 0x564464103d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 4 client.17280 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:10.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.132+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56446405d9d0 con 0x564464101900 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.133+0000 7fb50effd640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564464101900 0x564464103d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.133+0000 7fb4eeffd640 10 client.17280 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.133+0000 7fb50effd640 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564464101900 0x564464103d70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cbc124a4bd245a server_cookie=b302ace61d42714b in_seq=0 out_seq=0 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb4eeffd640 1 -- 172.21.15.96:0/2788277767 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56446405d9d0 con 0x564464101900 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb4eeffd640 10 client.17280 handle_command_reply: tid=0 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 client.17280 shutdown 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 msgr2=0x5644640fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb50801b040 tx=0x7fb508009d40 comp rx=0 tx=0).stop 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 shutdown_connections 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564464101900 0x564464103d70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640f13b0 0x5644640fd290 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 --2- 172.21.15.96:0/2788277767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640eb850 0x5644640f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 >> 172.21.15.96:0/2788277767 conn(0x5644640e6dc0 msgr2=0x5644640e5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb4eeffd640 0 client.17280 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:10.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 shutdown_connections 2024-03-18T15:47:10.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 1 -- 172.21.15.96:0/2788277767 wait complete. 2024-03-18T15:47:10.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.134+0000 7fb5185f9740 20 client.17280 trim_cache size 0 max 16384 2024-03-18T15:47:10.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.147+0000 7fb5185f9740 1 Processor -- start 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.147+0000 7fb5185f9740 1 -- start start 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101a8e10 con 0x5644640eb850 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101aa9e0 con 0x5644640e6dc0 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb5185f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101a9510 con 0x5644640ec240 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56352/0 (socket says 172.21.15.96:56352) 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 -- 172.21.15.96:0/152156720 learned_addr learned my addr 172.21.15.96:0/152156720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50effd640 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 -- 172.21.15.96:0/152156720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 msgr2=0x5644640e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640e5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 -- 172.21.15.96:0/152156720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 msgr2=0x5644640e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 -- 172.21.15.96:0/152156720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564463e2e7d0 con 0x5644640eb850 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb50400f870 tx=0x7fb50401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a110d4ff2bdec11d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb4f57fa640 1 -- 172.21.15.96:0/152156720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504017040 con 0x5644640eb850 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb4f57fa640 1 -- 172.21.15.96:0/152156720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504019050 con 0x5644640eb850 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.148+0000 7fb4f57fa640 1 -- 172.21.15.96:0/152156720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50401d440 con 0x5644640eb850 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- 172.21.15.96:0/152156720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 msgr2=0x5644640fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb50400f870 tx=0x7fb50401d000 comp rx=0 tx=0).stop 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- 172.21.15.96:0/152156720 shutdown_connections 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640ec240 0x5644640e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640eb850 0x5644640fd250 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 --2- 172.21.15.96:0/152156720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- 172.21.15.96:0/152156720 >> 172.21.15.96:0/152156720 conn(0x5644640f56e0 msgr2=0x5644640f5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- 172.21.15.96:0/152156720 shutdown_connections 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- 172.21.15.96:0/152156720 wait complete. 2024-03-18T15:47:10.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 -- start start 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.149+0000 7fb5185f9740 1 Processor -- start 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101aa310 con 0x5644640ec240 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101a9c10 con 0x5644640e6dc0 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5101a6df0 con 0x5644640eb850 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 10 client.0 _ll_register_callbacks cb 0x7fb5040099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56368/0 (socket says 172.21.15.96:56368) 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 -- 172.21.15.96:0/1421017515 learned_addr learned my addr 172.21.15.96:0/1421017515 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50effd640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50e7fc640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 -- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 msgr2=0x5644640fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 -- 172.21.15.96:0/1421017515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 msgr2=0x5644640fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640fa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 -- 172.21.15.96:0/1421017515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644640fbfd0 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb50f7fe640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb508003d60 tx=0x7fb508009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63742f03b3a60fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb4eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080052b0 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb508005450 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.150+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb504019050 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb5080228c0 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 10 client.17286 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5644640fd2d0 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb504019050 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fb50800f5c0 con 0x5644640ec240 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 10 client.17286 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:10.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 20 client.17286 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5644640f13b0 0x56446416a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 4 client.17286 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564464057d20 con 0x5644640f13b0 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb50effd640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5644640f13b0 0x56446416a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb4eeffd640 10 client.17286 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.151+0000 7fb50effd640 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5644640f13b0 0x56446416a1c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf56c82e17bd80 server_cookie=abfaab937c44d0bc in_seq=0 out_seq=0 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb4eeffd640 1 -- 172.21.15.96:0/1421017515 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564464057d20 con 0x5644640f13b0 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb4eeffd640 10 client.17286 handle_command_reply: tid=0 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 client.17286 shutdown 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 msgr2=0x5644640fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb508003d60 tx=0x7fb508009db0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 shutdown_connections 2024-03-18T15:47:10.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5644640f13b0 0x56446416a1c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5644640ec240 0x5644640fb9e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644640eb850 0x5644640fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb4eeffd640 0 client.17286 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 --2- 172.21.15.96:0/1421017515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644640e6dc0 0x5644640fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 >> 172.21.15.96:0/1421017515 conn(0x5644640f56e0 msgr2=0x5644640ea020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 shutdown_connections 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 1 -- 172.21.15.96:0/1421017515 wait complete. 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb5185f9740 20 client.17286 trim_cache size 0 max 16384 2024-03-18T15:47:10.160 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:10.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 msgr2=0x7fb4f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 0x7fb4f80a4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb5101990a0 tx=0x7fb504011440 comp rx=0 tx=0).stop 2024-03-18T15:47:10.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 msgr2=0x7fb510197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.152+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb500000f80 tx=0x7fb500032ec0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 shutdown_connections 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4f80a2800 0x7fb4f80a4cc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb510102290 0x7fb51019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5101018d0 0x7fb5101980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 --2- 172.21.15.96:0/1283858845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5101006e0 0x7fb510197b90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 >> 172.21.15.96:0/1283858845 conn(0x7fb5100fbc70 msgr2=0x7fb510068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 shutdown_connections 2024-03-18T15:47:10.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:10.153+0000 7fb515a0b640 1 -- 172.21.15.96:0/1283858845 wait complete. 2024-03-18T15:47:10.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:10 smithi193 ceph-mon[36702]: pgmap v565: 97 pgs: 97 active+clean; 547 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 975 KiB/s wr, 541 op/s 2024-03-18T15:47:10.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:10 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:10 smithi096 ceph-mon[31930]: pgmap v565: 97 pgs: 97 active+clean; 547 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 975 KiB/s wr, 541 op/s 2024-03-18T15:47:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:10 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:11.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:10 smithi137 ceph-mon[35868]: pgmap v565: 97 pgs: 97 active+clean; 547 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 975 KiB/s wr, 541 op/s 2024-03-18T15:47:11.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:10 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:11.165 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:47:11.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:47:11.165 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:47:11.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- 172.21.15.96:0/1059688876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01006e0 msgr2=0x7fc0e0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- 172.21.15.96:0/1059688876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01006e0 0x7fc0e0100b00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009900 tx=0x7fc0d002bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- 172.21.15.96:0/1059688876 shutdown_connections 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- 172.21.15.96:0/1059688876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 0x7fc0e0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- 172.21.15.96:0/1059688876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01018d0 0x7fc0e0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- 172.21.15.96:0/1059688876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01006e0 0x7fc0e0100b00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- 172.21.15.96:0/1059688876 >> 172.21.15.96:0/1059688876 conn(0x7fc0e00fbc70 msgr2=0x7fc0e00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- 172.21.15.96:0/1059688876 shutdown_connections 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- 172.21.15.96:0/1059688876 wait complete. 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 Processor -- start 2024-03-18T15:47:11.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- start start 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01006e0 0x7fc0e0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 0x7fc0e019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010d540 con 0x7fc0e01018d0 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010d3c0 con 0x7fc0e01006e0 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0e535b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e010d6c0 con 0x7fc0e0102290 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.274+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56386/0 (socket says 172.21.15.96:56386) 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 -- 172.21.15.96:0/2287632669 learned_addr learned my addr 172.21.15.96:0/2287632669 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 0x7fc0e019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0deffd640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01006e0 0x7fc0e0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 -- 172.21.15.96:0/2287632669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 msgr2=0x7fc0e019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 0x7fc0e019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 -- 172.21.15.96:0/2287632669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01006e0 msgr2=0x7fc0e0197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01006e0 0x7fc0e0197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 -- 172.21.15.96:0/2287632669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e019f850 con 0x7fc0e01018d0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b4a0 tx=0x7fc0c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0dcff9640 1 -- 172.21.15.96:0/2287632669 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014070 con 0x7fc0e01018d0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0dcff9640 1 -- 172.21.15.96:0/2287632669 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8010af0 con 0x7fc0e01018d0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019fb40 con 0x7fc0e01018d0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0dcff9640 1 -- 172.21.15.96:0/2287632669 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8017d10 con 0x7fc0e01018d0 2024-03-18T15:47:11.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.275+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e01a0060 con 0x7fc0e01018d0 2024-03-18T15:47:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0dcff9640 1 -- 172.21.15.96:0/2287632669 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc0c8017070 con 0x7fc0e01018d0 2024-03-18T15:47:11.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 Processor -- start 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 -- start start 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 0x56350eb3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb3b4f0 0x56350eb3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56350ea9a0d0 con 0x56350eb35990 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56350ea42c70 con 0x56350eb36380 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0e8026740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56350ea96170 con 0x56350eb3b4f0 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0dcff9640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 0x7fc0cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.278+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 0x56350eb3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56390/0 (socket says 172.21.15.96:56390) 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0df7fe640 1 -- 172.21.15.96:0/3252429453 learned_addr learned my addr 172.21.15.96:0/3252429453 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb3b4f0 0x56350eb3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0dcff9640 1 -- 172.21.15.96:0/2287632669 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc0c80cdda0 con 0x7fc0e01018d0 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 0x7fc0cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 -- 172.21.15.96:0/3252429453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb3b4f0 msgr2=0x56350eb3d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb3b4f0 0x56350eb3d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 -- 172.21.15.96:0/3252429453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 msgr2=0x56350eb35db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 -- 172.21.15.96:0/3252429453 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350e8767e0 con 0x56350eb36380 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0deffd640 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 0x56350eb3ae80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0d002c170 tx=0x7fc0d0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a50d0f5764a641c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 0x7fc0cc0a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x56350eb36be0 tx=0x7fc0d8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0d57fa640 1 -- 172.21.15.96:0/3252429453 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0042070 con 0x56350eb36380 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0d57fa640 1 -- 172.21.15.96:0/3252429453 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d00046c0 con 0x56350eb36380 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.279+0000 7fc0d57fa640 1 -- 172.21.15.96:0/3252429453 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0005380 con 0x56350eb36380 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- 172.21.15.96:0/3252429453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 msgr2=0x56350eb3ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 0x56350eb3ae80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0d002c170 tx=0x7fc0d0005dd0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- 172.21.15.96:0/3252429453 shutdown_connections 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb3b4f0 0x56350eb3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb36380 0x56350eb3ae80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3252429453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- 172.21.15.96:0/3252429453 >> 172.21.15.96:0/3252429453 conn(0x56350eafdc50 msgr2=0x56350eb31300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- 172.21.15.96:0/3252429453 shutdown_connections 2024-03-18T15:47:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- 172.21.15.96:0/3252429453 wait complete. 2024-03-18T15:47:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 -- start start 2024-03-18T15:47:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.280+0000 7fc0e8026740 1 Processor -- start 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56350eb3e440 con 0x56350eb3b4f0 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56350ea560d0 con 0x56350eb35990 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56350eb3e5c0 con 0x56350eb36380 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 10 client.0 _ll_register_callbacks cb 0x56350ea1f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56404/0 (socket says 172.21.15.96:56404) 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb3fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55470/0 (socket says 172.21.15.96:55470) 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 -- 172.21.15.96:0/3090276508 learned_addr learned my addr 172.21.15.96:0/3090276508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 -- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 msgr2=0x56350eb403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb403b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 -- 172.21.15.96:0/3090276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 msgr2=0x56350eb3fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb3fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 -- 172.21.15.96:0/3090276508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350e8d6f70 con 0x56350eb3b4f0 2024-03-18T15:47:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80098c0 tx=0x7fc0d8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155bb5979a6f5bda server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80233f0 con 0x56350eb3b4f0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc0d8023590 con 0x56350eb3b4f0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.281+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d00046c0 con 0x56350eb3b4f0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0d8023790 con 0x56350eb3b4f0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 10 client.17298 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56350e76e170 con 0x56350eb3b4f0 2024-03-18T15:47:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d00046c0 con 0x56350eb3b4f0 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fc0d8023930 con 0x56350eb3b4f0 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 10 client.17298 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 20 client.17298 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.282+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350eb4ba50 0x56350eb4df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.283+0000 7fc0e8026740 4 client.17298 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.283+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350eaa7610 con 0x56350eb4ba50 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.283+0000 7fc0deffd640 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350eb4ba50 0x56350eb4df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.283+0000 7fc0c67fc640 10 client.17298 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.283+0000 7fc0deffd640 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350eb4ba50 0x56350eb4df10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349bd2a8f1d8191d server_cookie=ee0486b49e4113f5 in_seq=0 out_seq=0 2024-03-18T15:47:11.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0c67fc640 1 -- 172.21.15.96:0/3090276508 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x56350eaa7610 con 0x56350eb4ba50 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0c67fc640 10 client.17298 handle_command_reply: tid=0 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 client.17298 shutdown 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 msgr2=0x56350eb473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80098c0 tx=0x7fc0d8004b50 comp rx=0 tx=0).stop 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 shutdown_connections 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350eb4ba50 0x56350eb4df10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb473f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0c67fc640 0 client.17298 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 --2- 172.21.15.96:0/3090276508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb3fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 >> 172.21.15.96:0/3090276508 conn(0x56350eafdc50 msgr2=0x56350eb304e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 shutdown_connections 2024-03-18T15:47:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 1 -- 172.21.15.96:0/3090276508 wait complete. 2024-03-18T15:47:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.284+0000 7fc0e8026740 20 client.17298 trim_cache size 0 max 16384 2024-03-18T15:47:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 Processor -- start 2024-03-18T15:47:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 -- start start 2024-03-18T15:47:11.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a8e10 con 0x56350eb3b4f0 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01aa9e0 con 0x56350eb35990 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0e8026740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a9510 con 0x56350eb36380 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:55486/0 (socket says 172.21.15.96:55486) 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0deffd640 1 -- 172.21.15.96:0/450638541 learned_addr learned my addr 172.21.15.96:0/450638541 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 -- 172.21.15.96:0/450638541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 msgr2=0x56350eb473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 -- 172.21.15.96:0/450638541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 msgr2=0x56350eb34c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0df7fe640 1 -- 172.21.15.96:0/450638541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350eb35190 con 0x56350eb3b4f0 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0deffd640 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.302+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb527d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8023df0 tx=0x7fc0d8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e868343c80f7746 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0d4ff9640 1 -- 172.21.15.96:0/450638541 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d803b070 con 0x56350eb3b4f0 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0d4ff9640 1 -- 172.21.15.96:0/450638541 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d8028070 con 0x56350eb3b4f0 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0d4ff9640 1 -- 172.21.15.96:0/450638541 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8023b90 con 0x56350eb3b4f0 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 -- 172.21.15.96:0/450638541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 msgr2=0x56350eb527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb527d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8023df0 tx=0x7fc0d8009f00 comp rx=0 tx=0).stop 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 -- 172.21.15.96:0/450638541 shutdown_connections 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb3b4f0 0x56350eb527d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 --2- 172.21.15.96:0/450638541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb35990 0x56350eb34c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 -- 172.21.15.96:0/450638541 >> 172.21.15.96:0/450638541 conn(0x56350eafdc50 msgr2=0x56350eb34540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 -- 172.21.15.96:0/450638541 shutdown_connections 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.303+0000 7fc0e8026740 1 -- 172.21.15.96:0/450638541 wait complete. 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 -- start start 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 Processor -- start 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb3b4f0 0x56350eb45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01aa310 con 0x56350eb35990 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a9c10 con 0x56350eb3b4f0 2024-03-18T15:47:11.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01a6df0 con 0x56350eb36380 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0e8026740 10 client.0 _ll_register_callbacks cb 0x7fc0d8015cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56426/0 (socket says 172.21.15.96:56426) 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 -- 172.21.15.96:0/4072024933 learned_addr learned my addr 172.21.15.96:0/4072024933 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0df7fe640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb3b4f0 0x56350eb45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 -- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 msgr2=0x56350eb472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 -- 172.21.15.96:0/4072024933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb3b4f0 msgr2=0x56350eb45400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb3b4f0 0x56350eb45400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.304+0000 7fc0deffd640 1 -- 172.21.15.96:0/4072024933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350eb45c30 con 0x56350eb35990 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0deffd640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00096c0 tx=0x7fc0d002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ffd427e3aa3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0054060 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc0d004fb10 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d8028070 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc0d0042040 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 10 client.17304 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56350ebb2a10 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d8028070 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7fc0d0004160 con 0x56350eb35990 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 10 client.17304 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 20 client.17304 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350ebb2fd0 0x56350ebb53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 4 client.17304 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56350eaa1960 con 0x56350ebb2fd0 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.305+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350ebb2fd0 0x56350ebb53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.306+0000 7fc0c67fc640 10 client.17304 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.306+0000 7fc0d7fff640 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350ebb2fd0 0x56350ebb53c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1610fe1d149c96 server_cookie=4b043afcd969d7ae in_seq=0 out_seq=0 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0c67fc640 1 -- 172.21.15.96:0/4072024933 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56350eaa1960 con 0x56350ebb2fd0 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0c67fc640 10 client.17304 handle_command_reply: tid=0 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 client.17304 shutdown 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 msgr2=0x56350eb46db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00096c0 tx=0x7fc0d002c5d0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 shutdown_connections 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56350eb3b4f0 0x56350eb45400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56350eb36380 0x56350eb472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x56350ebb2fd0 0x56350ebb53c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 --2- 172.21.15.96:0/4072024933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56350eb35990 0x56350eb46db0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0c67fc640 0 client.17304 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 >> 172.21.15.96:0/4072024933 conn(0x56350eafdc50 msgr2=0x56350eb2fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 shutdown_connections 2024-03-18T15:47:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 1 -- 172.21.15.96:0/4072024933 wait complete. 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e8026740 20 client.17304 trim_cache size 0 max 16384 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "d2ec4df5-1407-4fb2-880c-c9469c3531fa", 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:47:11.317 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:11.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 msgr2=0x7fc0cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 0x7fc0cc0a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x56350eb36be0 tx=0x7fc0d8004a40 comp rx=0 tx=0).stop 2024-03-18T15:47:11.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 msgr2=0x7fc0e0198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b4a0 tx=0x7fc0c800b960 comp rx=0 tx=0).stop 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 shutdown_connections 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc0cc0a2800 0x7fc0cc0a4cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0e0102290 0x7fc0e019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0e01018d0 0x7fc0e0198180 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 --2- 172.21.15.96:0/2287632669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc0e01006e0 0x7fc0e0197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 >> 172.21.15.96:0/2287632669 conn(0x7fc0e00fbc70 msgr2=0x7fc0e0104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 shutdown_connections 2024-03-18T15:47:11.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.307+0000 7fc0e535b640 1 -- 172.21.15.96:0/2287632669 wait complete. 2024-03-18T15:47:11.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:47:11.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 -- 172.21.15.96:0/39320374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81022a0 msgr2=0x7f46d8108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 --2- 172.21.15.96:0/39320374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81022a0 0x7f46d8108b30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 -- 172.21.15.96:0/39320374 shutdown_connections 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 --2- 172.21.15.96:0/39320374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81022a0 0x7f46d8108b30 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 --2- 172.21.15.96:0/39320374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81018e0 0x7f46d8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 --2- 172.21.15.96:0/39320374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81006f0 0x7f46d8100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 -- 172.21.15.96:0/39320374 >> 172.21.15.96:0/39320374 conn(0x7f46d80fbc80 msgr2=0x7f46d80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 -- 172.21.15.96:0/39320374 shutdown_connections 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.467+0000 7f46dde19640 1 -- 172.21.15.96:0/39320374 wait complete. 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 Processor -- start 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 -- start start 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81018e0 0x7f46d8196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81022a0 0x7f46d819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810c370 con 0x7f46d81006f0 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810c1f0 con 0x7f46d81022a0 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46dde19640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810c4f0 con 0x7f46d81018e0 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56440/0 (socket says 172.21.15.96:56440) 2024-03-18T15:47:11.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 -- 172.21.15.96:0/2286321284 learned_addr learned my addr 172.21.15.96:0/2286321284 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81018e0 0x7f46d8196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d7fff640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81022a0 0x7f46d819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 -- 172.21.15.96:0/2286321284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81018e0 msgr2=0x7f46d8196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81018e0 0x7f46d8196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 -- 172.21.15.96:0/2286321284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81022a0 msgr2=0x7f46d819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81022a0 0x7f46d819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.468+0000 7f46d77fe640 1 -- 172.21.15.96:0/2286321284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d819e5e0 con 0x7f46d81006f0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46d77fe640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f46c800e9c0 tx=0x7f46c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46d4ff9640 1 -- 172.21.15.96:0/2286321284 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c800cd80 con 0x7f46d81006f0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d819e8d0 con 0x7f46d81006f0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46d4ff9640 1 -- 172.21.15.96:0/2286321284 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c80035f0 con 0x7f46d81006f0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46d4ff9640 1 -- 172.21.15.96:0/2286321284 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8010690 con 0x7f46d81006f0 2024-03-18T15:47:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.469+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d819ee20 con 0x7f46d81006f0 2024-03-18T15:47:11.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.471+0000 7f46d4ff9640 1 -- 172.21.15.96:0/2286321284 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f46c80037a0 con 0x7f46d81006f0 2024-03-18T15:47:11.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.471+0000 7f46e0a08740 1 Processor -- start 2024-03-18T15:47:11.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.471+0000 7f46e0a08740 1 -- start start 2024-03-18T15:47:11.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe18c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe191660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563bfe0f1500 con 0x563bfe191cd0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563bfe099be0 con 0x563bfe18c170 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46e0a08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563bfe0ed5a0 con 0x563bfe18cb60 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d4ff9640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 0x7f46c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe191660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56444/0 (socket says 172.21.15.96:56444) 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe18c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 -- 172.21.15.96:0/3584671860 learned_addr learned my addr 172.21.15.96:0/3584671860 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d7fff640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 0x7f46c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d4ff9640 1 -- 172.21.15.96:0/2286321284 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f46c80cedf0 con 0x7f46d81006f0 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 -- 172.21.15.96:0/3584671860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 msgr2=0x563bfe191660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe191660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 -- 172.21.15.96:0/3584671860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 msgr2=0x563bfe18c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe18c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 -- 172.21.15.96:0/3584671860 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfe0ad4b0 con 0x563bfe191cd0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d7fff640 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe18c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d7fff640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 0x7f46c00a4cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46d0005e30 tx=0x7f46d0032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009c90 tx=0x7f46cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9f5301018b4831 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46bdffb640 1 -- 172.21.15.96:0/3584671860 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc01e070 con 0x563bfe191cd0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.472+0000 7f46bdffb640 1 -- 172.21.15.96:0/3584671860 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc019b10 con 0x563bfe191cd0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46bdffb640 1 -- 172.21.15.96:0/3584671860 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00eae0 con 0x563bfe191cd0 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 -- 172.21.15.96:0/3584671860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 msgr2=0x563bfe194160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009c90 tx=0x7f46cc011810 comp rx=0 tx=0).stop 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 -- 172.21.15.96:0/3584671860 shutdown_connections 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe194160 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe191660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3584671860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe18c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 -- 172.21.15.96:0/3584671860 >> 172.21.15.96:0/3584671860 conn(0x563bfe1876e0 msgr2=0x563bfe187ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 -- 172.21.15.96:0/3584671860 shutdown_connections 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.473+0000 7f46e0a08740 1 -- 172.21.15.96:0/3584671860 wait complete. 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 -- start start 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 Processor -- start 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563bfe194c80 con 0x563bfe18c170 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563bfe0ad160 con 0x563bfe18cb60 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563bfe194e00 con 0x563bfe191cd0 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46e0a08740 10 client.0 _ll_register_callbacks cb 0x563bfe091150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56446/0 (socket says 172.21.15.96:56446) 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 -- 172.21.15.96:0/1538345091 learned_addr learned my addr 172.21.15.96:0/1538345091 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d7fff640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 -- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 msgr2=0x563bfe19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 -- 172.21.15.96:0/1538345091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 msgr2=0x563bfe196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.474+0000 7f46d77fe640 1 -- 172.21.15.96:0/1538345091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bfdecf7d0 con 0x563bfe18c170 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f46c806e460 tx=0x7f46c806d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad770df910933419 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b4060 con 0x563bfe18c170 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f46c8003b10 con 0x563bfe18c170 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46cc019b10 con 0x563bfe18c170 2024-03-18T15:47:11.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f46c809b0c0 con 0x563bfe18c170 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 10 client.17316 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563bfde053f0 con 0x563bfe18c170 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc019b10 con 0x563bfe18c170 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f46c806c040 con 0x563bfe18c170 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 10 client.17316 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 20 client.17316 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe1a2160 0x563bfe1a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 4 client.17316 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.475+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bfe0fea40 con 0x563bfe1a2160 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.476+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe1a2160 0x563bfe1a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.476+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe1a2160 0x563bfe1a4620 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f593078a395fdb92 server_cookie=b33127ba738cb068 in_seq=0 out_seq=0 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.476+0000 7f46b6ffd640 10 client.17316 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1538345091 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563bfe0fea40 con 0x563bfe1a2160 2024-03-18T15:47:11.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46b6ffd640 10 client.17316 handle_command_reply: tid=0 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 client.17316 shutdown 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 msgr2=0x563bfe196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f46c806e460 tx=0x7f46c806d3e0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 shutdown_connections 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe1a2160 0x563bfe1a4620 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1538345091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe196660 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46b6ffd640 0 client.17316 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 >> 172.21.15.96:0/1538345091 conn(0x563bfe1876e0 msgr2=0x563bfe186900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 shutdown_connections 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 1 -- 172.21.15.96:0/1538345091 wait complete. 2024-03-18T15:47:11.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.477+0000 7f46e0a08740 20 client.17316 trim_cache size 0 max 16384 2024-03-18T15:47:11.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 Processor -- start 2024-03-18T15:47:11.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 -- start start 2024-03-18T15:47:11.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe196b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe19db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a6b10 con 0x563bfe191cd0 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a86e0 con 0x563bfe18c170 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.495+0000 7f46e0a08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a7210 con 0x563bfe18cb60 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56462/0 (socket says 172.21.15.96:56462) 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 -- 172.21.15.96:0/3957738187 learned_addr learned my addr 172.21.15.96:0/3957738187 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d77fe640 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 -- 172.21.15.96:0/3957738187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 msgr2=0x563bfe19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 -- 172.21.15.96:0/3957738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 msgr2=0x563bfe196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 -- 172.21.15.96:0/3957738187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfddc50f0 con 0x563bfe191cd0 2024-03-18T15:47:11.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46d7fff640 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fad8b10f6554ea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46bd7fa640 1 -- 172.21.15.96:0/3957738187 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d0039040 con 0x563bfe191cd0 2024-03-18T15:47:11.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46bd7fa640 1 -- 172.21.15.96:0/3957738187 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d0054070 con 0x563bfe191cd0 2024-03-18T15:47:11.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.496+0000 7f46bd7fa640 1 -- 172.21.15.96:0/3957738187 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d004f6c0 con 0x563bfe191cd0 2024-03-18T15:47:11.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 -- 172.21.15.96:0/3957738187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 msgr2=0x563bfe18b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ca0 tx=0x7f46d002e040 comp rx=0 tx=0).stop 2024-03-18T15:47:11.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 -- 172.21.15.96:0/3957738187 shutdown_connections 2024-03-18T15:47:11.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe191cd0 0x563bfe18b050 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe18cb60 0x563bfe19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 --2- 172.21.15.96:0/3957738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18c170 0x563bfe196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 -- 172.21.15.96:0/3957738187 >> 172.21.15.96:0/3957738187 conn(0x563bfe1876e0 msgr2=0x563bfe18a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 -- 172.21.15.96:0/3957738187 shutdown_connections 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.497+0000 7f46e0a08740 1 -- 172.21.15.96:0/3957738187 wait complete. 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 -- start start 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 Processor -- start 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe19b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a8010 con 0x563bfe18c170 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a7910 con 0x563bfe18cb60 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81a4af0 con 0x563bfe191cd0 2024-03-18T15:47:11.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46e0a08740 10 client.0 _ll_register_callbacks cb 0x7f46c80096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56466/0 (socket says 172.21.15.96:56466) 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46d77fe640 1 -- 172.21.15.96:0/1116375439 learned_addr learned my addr 172.21.15.96:0/1116375439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.498+0000 7f46d7fff640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 -- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 msgr2=0x563bfe19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 -- 172.21.15.96:0/1116375439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 msgr2=0x563bfe19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 -- 172.21.15.96:0/1116375439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bfe0b89d0 con 0x563bfe18c170 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46d77fe640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f46c8097df0 tx=0x7f46c80862e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb6a9ad85a1963a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b4040 con 0x563bfe18c170 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f46c80cc2f0 con 0x563bfe18c170 2024-03-18T15:47:11.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d0054070 con 0x563bfe18c170 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f46c8097e20 con 0x563bfe18c170 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46e0a08740 10 client.17322 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563bfe19de00 con 0x563bfe18c170 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d0054070 con 0x563bfe18c170 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.499+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f46c80afb40 con 0x563bfe18c170 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 10 client.17322 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 20 client.17322 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe208af0 0x563bfe20af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 4 client.17322 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563bfe0f8d90 con 0x563bfe208af0 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe208af0 0x563bfe20af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46d6ffd640 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe208af0 0x563bfe20af10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3273d8c8dd6c3c27 server_cookie=4bf22e8861d2a175 in_seq=0 out_seq=0 2024-03-18T15:47:11.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.500+0000 7f46b6ffd640 10 client.17322 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46b6ffd640 1 -- 172.21.15.96:0/1116375439 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563bfe0f8d90 con 0x563bfe208af0 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46b6ffd640 10 client.17322 handle_command_reply: tid=0 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 client.17322 shutdown 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 msgr2=0x563bfe19b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f46c8097df0 tx=0x7f46c80862e0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 shutdown_connections 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563bfe191cd0 0x563bfe19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563bfe18cb60 0x563bfe19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x563bfe208af0 0x563bfe20af10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 --2- 172.21.15.96:0/1116375439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563bfe18c170 0x563bfe19b110 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46b6ffd640 0 client.17322 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 >> 172.21.15.96:0/1116375439 conn(0x563bfe1876e0 msgr2=0x563bfe18b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 shutdown_connections 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 1 -- 172.21.15.96:0/1116375439 wait complete. 2024-03-18T15:47:11.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46e0a08740 20 client.17322 trim_cache size 0 max 16384 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "d2ec4df5-1407-4fb2-880c-c9469c3531fa": { 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "d2ec4df5-1407-4fb2-880c-c9469c3531fa", 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:47:11.508 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:11.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 msgr2=0x7f46c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 0x7f46c00a4cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f46d0005e30 tx=0x7f46d0032590 comp rx=0 tx=0).stop 2024-03-18T15:47:11.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 msgr2=0x7f46d81969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:11.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.501+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f46c800e9c0 tx=0x7f46c800ee80 comp rx=0 tx=0).stop 2024-03-18T15:47:11.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 shutdown_connections 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f46c00a2800 0x7f46c00a4cc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f46d81022a0 0x7f46d819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46d81018e0 0x7f46d8196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 --2- 172.21.15.96:0/2286321284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46d81006f0 0x7f46d81969c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 >> 172.21.15.96:0/2286321284 conn(0x7f46d80fbc80 msgr2=0x7f46d8104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 shutdown_connections 2024-03-18T15:47:11.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:11.502+0000 7f46dde19640 1 -- 172.21.15.96:0/2286321284 wait complete. 2024-03-18T15:47:11.515 INFO:tasks.cephfs.filesystem:scrub status for tag:d2ec4df5-1407-4fb2-880c-c9469c3531fa - {'path': '/', 'tag': 'd2ec4df5-1407-4fb2-880c-c9469c3531fa', 'options': 'recursive,force'} 2024-03-18T15:47:11.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:11 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:47:11.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:11 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:11.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:11 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:11.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:11 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:11 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:47:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:11 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:11 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:12.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:11 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:11 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:47:12.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:11 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:11 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:12.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:11 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: pgmap v566: 97 pgs: 97 active+clean; 546 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 852 KiB/s wr, 398 op/s 2024-03-18T15:47:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:12.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:12 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: pgmap v566: 97 pgs: 97 active+clean; 546 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 852 KiB/s wr, 398 op/s 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:13.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:12 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: pgmap v566: 97 pgs: 97 active+clean; 546 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 852 KiB/s wr, 398 op/s 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:13.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:12 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 -- 172.21.15.96:0/375680698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc100700 msgr2=0x7feafc100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 --2- 172.21.15.96:0/375680698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc100700 0x7feafc100b20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaec009900 tx=0x7feaec02bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 -- 172.21.15.96:0/375680698 shutdown_connections 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 --2- 172.21.15.96:0/375680698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc1022b0 0x7feafc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 --2- 172.21.15.96:0/375680698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1018f0 0x7feafc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 --2- 172.21.15.96:0/375680698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc100700 0x7feafc100b20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 -- 172.21.15.96:0/375680698 >> 172.21.15.96:0/375680698 conn(0x7feafc0fbc70 msgr2=0x7feafc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 -- 172.21.15.96:0/375680698 shutdown_connections 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.571+0000 7feb01a75640 1 -- 172.21.15.96:0/375680698 wait complete. 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 Processor -- start 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 -- start start 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc100700 0x7feafc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1022b0 0x7feafc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feafc10c350 con 0x7feafc1018f0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feafc10c1d0 con 0x7feafc1022b0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feb01a75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feafc10c4d0 con 0x7feafc100700 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feafaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feafaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53622/0 (socket says 172.21.15.96:53622) 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feafaffd640 1 -- 172.21.15.96:0/2740769966 learned_addr learned my addr 172.21.15.96:0/2740769966 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.572+0000 7feafb7fe640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc100700 0x7feafc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafbfff640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1022b0 0x7feafc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 -- 172.21.15.96:0/2740769966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc100700 msgr2=0x7feafc069980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc100700 0x7feafc069980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 -- 172.21.15.96:0/2740769966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1022b0 msgr2=0x7feafc19ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1022b0 0x7feafc19ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 -- 172.21.15.96:0/2740769966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc19f6e0 con 0x7feafc1018f0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feafaffd640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e9f0 tx=0x7feaf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feaf8ff9640 1 -- 172.21.15.96:0/2740769966 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000cd80 con 0x7feafc1018f0 2024-03-18T15:47:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feaf8ff9640 1 -- 172.21.15.96:0/2740769966 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0010910 con 0x7feafc1018f0 2024-03-18T15:47:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc19f9d0 con 0x7feafc1018f0 2024-03-18T15:47:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feaf8ff9640 1 -- 172.21.15.96:0/2740769966 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0018990 con 0x7feafc1018f0 2024-03-18T15:47:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.573+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc19ff70 con 0x7feafc1018f0 2024-03-18T15:47:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.575+0000 7feaf8ff9640 1 -- 172.21.15.96:0/2740769966 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7feaf0010ac0 con 0x7feafc1018f0 2024-03-18T15:47:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feaf8ff9640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 0x7feae40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 Processor -- start 2024-03-18T15:47:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 -- start start 2024-03-18T15:47:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafb7fe640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 0x7feae40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feaf8ff9640 1 -- 172.21.15.96:0/2740769966 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7feaf00ced20 con 0x7feafc1018f0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a8231b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a8239740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609a8192a40 con 0x5609a8232140 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5609a813b920 con 0x5609a82372b0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feb0465a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609a818eae0 con 0x5609a8231750 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53634/0 (socket says 172.21.15.96:53634) 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 -- 172.21.15.96:0/3231977498 learned_addr learned my addr 172.21.15.96:0/3231977498 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafb7fe640 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a8231b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafbfff640 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a8239740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafb7fe640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 0x7feae40a4de0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feaec009230 tx=0x7feaec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 -- 172.21.15.96:0/3231977498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 msgr2=0x5609a8231b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a8231b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 -- 172.21.15.96:0/3231977498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 msgr2=0x5609a8239740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a8239740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.576+0000 7feafaffd640 1 -- 172.21.15.96:0/3231977498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8153460 con 0x5609a8232140 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feafaffd640 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feaf009b6d0 tx=0x7feaf009b1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021fcbf4e2d9b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feae1ffb640 1 -- 172.21.15.96:0/3231977498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0099a40 con 0x5609a8232140 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feae1ffb640 1 -- 172.21.15.96:0/3231977498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf00860b0 con 0x5609a8232140 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feae1ffb640 1 -- 172.21.15.96:0/3231977498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf009adf0 con 0x5609a8232140 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 -- 172.21.15.96:0/3231977498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 msgr2=0x5609a8236c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feaf009b6d0 tx=0x7feaf009b1c0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 -- 172.21.15.96:0/3231977498 shutdown_connections 2024-03-18T15:47:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a8239740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8232140 0x5609a8236c40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 --2- 172.21.15.96:0/3231977498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a8231b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 -- 172.21.15.96:0/3231977498 >> 172.21.15.96:0/3231977498 conn(0x5609a82288b0 msgr2=0x5609a8228cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 -- 172.21.15.96:0/3231977498 shutdown_connections 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.577+0000 7feb0465a740 1 -- 172.21.15.96:0/3231977498 wait complete. 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 -- start start 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 Processor -- start 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8232140 0x5609a823bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609a823a260 con 0x5609a82372b0 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5609a814ed70 con 0x5609a8232140 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5609a823a3e0 con 0x5609a8231750 2024-03-18T15:47:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feb0465a740 10 client.0 _ll_register_callbacks cb 0x5609a7ea6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feafb7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53642/0 (socket says 172.21.15.96:53642) 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feafbfff640 1 -- 172.21.15.96:0/2560180006 learned_addr learned my addr 172.21.15.96:0/2560180006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.578+0000 7feafaffd640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8232140 0x5609a823bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 -- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 msgr2=0x5609a823ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 -- 172.21.15.96:0/2560180006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8232140 msgr2=0x5609a823bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8232140 0x5609a823bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 -- 172.21.15.96:0/2560180006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a7ea7210 con 0x5609a82372b0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feafbfff640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feaf4003c90 tx=0x7feaf4004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa65b23a6badb0f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4011650 con 0x5609a82372b0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7feaf40117f0 con 0x5609a82372b0 2024-03-18T15:47:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf00860b0 con 0x5609a82372b0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7feaf40119f0 con 0x5609a82372b0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 10 client.17334 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609a7f73620 con 0x5609a82372b0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf00860b0 con 0x5609a82372b0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7feaf401b420 con 0x5609a82372b0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 10 client.17334 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 20 client.17334 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a8247a40 0x5609a8249e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 4 client.17334 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.579+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a819ff80 con 0x5609a8247a40 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.580+0000 7feafb7fe640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a8247a40 0x5609a8249e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.580+0000 7feafb7fe640 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a8247a40 0x5609a8249e30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92e6f93aa887c75 server_cookie=956f3e3c9f0357d0 in_seq=0 out_seq=0 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.580+0000 7feadaffd640 10 client.17334 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feadaffd640 1 -- 172.21.15.96:0/2560180006 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5609a819ff80 con 0x5609a8247a40 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feadaffd640 10 client.17334 handle_command_reply: tid=0 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 client.17334 shutdown 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 msgr2=0x5609a8242fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feaf4003c90 tx=0x7feaf4004410 comp rx=0 tx=0).stop 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 shutdown_connections 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a8247a40 0x5609a8249e30 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a8242fd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8232140 0x5609a823bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 --2- 172.21.15.96:0/2560180006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feadaffd640 0 client.17334 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 >> 172.21.15.96:0/2560180006 conn(0x5609a82288b0 msgr2=0x5609a8227bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 shutdown_connections 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 1 -- 172.21.15.96:0/2560180006 wait complete. 2024-03-18T15:47:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.581+0000 7feb0465a740 20 client.17334 trim_cache size 0 max 16384 2024-03-18T15:47:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.596+0000 7feb0465a740 1 Processor -- start 2024-03-18T15:47:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.596+0000 7feb0465a740 1 -- start start 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.596+0000 7feb0465a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.596+0000 7feb0465a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8242f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.596+0000 7feb0465a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a823bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feb0465a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a6b60 con 0x5609a8231750 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feb0465a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a8730 con 0x5609a82372b0 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feb0465a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a7260 con 0x5609a8232140 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53646/0 (socket says 172.21.15.96:53646) 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 -- 172.21.15.96:0/1621047104 learned_addr learned my addr 172.21.15.96:0/1621047104 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafaffd640 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8242f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafbfff640 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a823bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.602 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 -- 172.21.15.96:0/1621047104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 msgr2=0x5609a8242f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8242f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 -- 172.21.15.96:0/1621047104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 msgr2=0x5609a823bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a823bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 -- 172.21.15.96:0/1621047104 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a823a520 con 0x5609a8231750 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feafb7fe640 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feaec02c7b0 tx=0x7feaec040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f41eac10442e05 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feae17fa640 1 -- 172.21.15.96:0/1621047104 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec05b070 con 0x5609a8231750 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feae17fa640 1 -- 172.21.15.96:0/1621047104 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaec0055e0 con 0x5609a8231750 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.597+0000 7feae17fa640 1 -- 172.21.15.96:0/1621047104 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec034640 con 0x5609a8231750 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- 172.21.15.96:0/1621047104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 msgr2=0x5609a823ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7feaec02c7b0 tx=0x7feaec040000 comp rx=0 tx=0).stop 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- 172.21.15.96:0/1621047104 shutdown_connections 2024-03-18T15:47:14.603 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a82372b0 0x5609a823bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8242f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 --2- 172.21.15.96:0/1621047104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a8231750 0x5609a823ba50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- 172.21.15.96:0/1621047104 >> 172.21.15.96:0/1621047104 conn(0x5609a82288b0 msgr2=0x5609a8223a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- 172.21.15.96:0/1621047104 shutdown_connections 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- 172.21.15.96:0/1621047104 wait complete. 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 -- start start 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.598+0000 7feb0465a740 1 Processor -- start 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8231750 0x5609a8240d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8241290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a8060 con 0x5609a82372b0 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a7960 con 0x5609a8231750 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1a4b40 con 0x5609a8232140 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 10 client.0 _ll_register_callbacks cb 0x7feaf4009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.604 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53656/0 (socket says 172.21.15.96:53656) 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 -- 172.21.15.96:0/250125906 learned_addr learned my addr 172.21.15.96:0/250125906 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafb7fe640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8231750 0x5609a8240d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafaffd640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8241290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 -- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 msgr2=0x5609a8241290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8241290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 -- 172.21.15.96:0/250125906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8231750 msgr2=0x5609a8240d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8231750 0x5609a8240d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 -- 172.21.15.96:0/250125906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8242890 con 0x5609a82372b0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feafbfff640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f80 tx=0x7feaf4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22920ad0b9fc38a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf402b040 con 0x5609a82372b0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7feaf401b4e0 con 0x5609a82372b0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaec0055e0 con 0x5609a82372b0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7feaf4011b90 con 0x5609a82372b0 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 10 client.17340 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:14.605 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.599+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5609a82ae370 con 0x5609a82372b0 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaec0055e0 con 0x5609a82372b0 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7feaf403a070 con 0x5609a82372b0 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 10 client.17340 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 20 client.17340 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a82aed10 0x5609a82b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 4 client.17340 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609a8237020 con 0x5609a82aed10 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feafb7fe640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a82aed10 0x5609a82b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feadaffd640 10 client.17340 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.600+0000 7feafb7fe640 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a82aed10 0x5609a82b1130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5122b4763698437 server_cookie=21da806418dceeee in_seq=0 out_seq=0 2024-03-18T15:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feadaffd640 1 -- 172.21.15.96:0/250125906 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609a8237020 con 0x5609a82aed10 2024-03-18T15:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feadaffd640 10 client.17340 handle_command_reply: tid=0 2024-03-18T15:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 client.17340 shutdown 2024-03-18T15:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 msgr2=0x5609a82422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feaf4000f80 tx=0x7feaf4009cb0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 shutdown_connections 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5609a82aed10 0x5609a82b1130 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5609a82372b0 0x5609a82422a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feadaffd640 0 client.17340 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5609a8232140 0x5609a8241290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 --2- 172.21.15.96:0/250125906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5609a8231750 0x5609a8240d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 >> 172.21.15.96:0/250125906 conn(0x5609a82288b0 msgr2=0x5609a8233460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 shutdown_connections 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 1 -- 172.21.15.96:0/250125906 wait complete. 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.610+0000 7feb0465a740 20 client.17340 trim_cache size 0 max 16384 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:47:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 msgr2=0x7feae40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 0x7feae40a4de0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feaec009230 tx=0x7feaec000bc0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 msgr2=0x7feafc19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e9f0 tx=0x7feaf000eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 shutdown_connections 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feae40a2920 0x7feae40a4de0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feafc1022b0 0x7feafc19ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feafc1018f0 0x7feafc19c850 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 --2- 172.21.15.96:0/2740769966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feafc100700 0x7feafc069980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 >> 172.21.15.96:0/2740769966 conn(0x7feafc0fbc70 msgr2=0x7feafc104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 shutdown_connections 2024-03-18T15:47:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:14.611+0000 7feb01a75640 1 -- 172.21.15.96:0/2740769966 wait complete. 2024-03-18T15:47:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:14 smithi096 ceph-mon[31930]: pgmap v567: 97 pgs: 97 active+clean; 542 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-03-18T15:47:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:14 smithi137 ceph-mon[35868]: pgmap v567: 97 pgs: 97 active+clean; 542 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-03-18T15:47:15.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:14 smithi193 ceph-mon[36702]: pgmap v567: 97 pgs: 97 active+clean; 542 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-03-18T15:47:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:16 smithi096 ceph-mon[31930]: pgmap v568: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 884 KiB/s wr, 663 op/s 2024-03-18T15:47:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:16 smithi137 ceph-mon[35868]: pgmap v568: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 884 KiB/s wr, 663 op/s 2024-03-18T15:47:17.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:16 smithi193 ceph-mon[36702]: pgmap v568: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 884 KiB/s wr, 663 op/s 2024-03-18T15:47:18.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:18 smithi096 ceph-mon[31930]: pgmap v569: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 763 KiB/s wr, 559 op/s 2024-03-18T15:47:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:18 smithi137 ceph-mon[35868]: pgmap v569: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 763 KiB/s wr, 559 op/s 2024-03-18T15:47:19.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:18 smithi193 ceph-mon[36702]: pgmap v569: 97 pgs: 97 active+clean; 530 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 763 KiB/s wr, 559 op/s 2024-03-18T15:47:20.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:20 smithi096 ceph-mon[31930]: pgmap v570: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 871 op/s 2024-03-18T15:47:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:20 smithi137 ceph-mon[35868]: pgmap v570: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 871 op/s 2024-03-18T15:47:21.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:20 smithi193 ceph-mon[36702]: pgmap v570: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 871 op/s 2024-03-18T15:47:22.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:22 smithi096 ceph-mon[31930]: pgmap v571: 97 pgs: 97 active+clean; 490 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-03-18T15:47:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:22 smithi137 ceph-mon[35868]: pgmap v571: 97 pgs: 97 active+clean; 490 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-03-18T15:47:23.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:22 smithi193 ceph-mon[36702]: pgmap v571: 97 pgs: 97 active+clean; 490 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 794 op/s 2024-03-18T15:47:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:24 smithi137 ceph-mon[35868]: pgmap v572: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 847 op/s 2024-03-18T15:47:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:24 smithi193 ceph-mon[36702]: pgmap v572: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 847 op/s 2024-03-18T15:47:25.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:24 smithi096 ceph-mon[31930]: pgmap v572: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 847 op/s 2024-03-18T15:47:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:26 smithi137 ceph-mon[35868]: pgmap v573: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 792 op/s 2024-03-18T15:47:27.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:26 smithi193 ceph-mon[36702]: pgmap v573: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 792 op/s 2024-03-18T15:47:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:26 smithi096 ceph-mon[31930]: pgmap v573: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 792 op/s 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- 172.21.15.96:0/837487452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c1022b0 msgr2=0x7f0b0c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/837487452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c108b40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009ca0 tx=0x7f0b0402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- 172.21.15.96:0/837487452 shutdown_connections 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/837487452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c108b40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/837487452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1018f0 0x7f0b0c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/837487452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c100700 0x7f0b0c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- 172.21.15.96:0/837487452 >> 172.21.15.96:0/837487452 conn(0x7f0b0c0fbc70 msgr2=0x7f0b0c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- 172.21.15.96:0/837487452 shutdown_connections 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- 172.21.15.96:0/837487452 wait complete. 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 Processor -- start 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.711+0000 7f0b11fa9640 1 -- start start 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c1018f0 0x7f0b0c1096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10d4a0 con 0x7f0b0c100700 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10d320 con 0x7f0b0c1018f0 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b11fa9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10d620 con 0x7f0b0c1022b0 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44696/0 (socket says 172.21.15.96:44696) 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 -- 172.21.15.96:0/1807589183 learned_addr learned my addr 172.21.15.96:0/1807589183 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c1018f0 0x7f0b0c1096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 -- 172.21.15.96:0/1807589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1022b0 msgr2=0x7f0b0c109c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c109c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 -- 172.21.15.96:0/1807589183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c1018f0 msgr2=0x7f0b0c1096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c1018f0 0x7f0b0c1096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 -- 172.21.15.96:0/1807589183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c10a350 con 0x7f0b0c100700 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b0b7fe640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00e9c0 tx=0x7f0afc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.712+0000 7f0b097fa640 1 -- 172.21.15.96:0/1807589183 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc00cd80 con 0x7f0b0c100700 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.713+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a1bf0 con 0x7f0b0c100700 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.713+0000 7f0b097fa640 1 -- 172.21.15.96:0/1807589183 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0afc0035f0 con 0x7f0b0c100700 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.713+0000 7f0b097fa640 1 -- 172.21.15.96:0/1807589183 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc010690 con 0x7f0b0c100700 2024-03-18T15:47:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.713+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a2100 con 0x7f0b0c100700 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b097fa640 1 -- 172.21.15.96:0/1807589183 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0afc0037a0 con 0x7f0b0c100700 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 Processor -- start 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 -- start start 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe010f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b097fa640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 0x7f0af80a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe0112330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.723 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bfe006fa40 con 0x55bfe010a340 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0018920 con 0x55bfe010ad30 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bfe006bae0 con 0x55bfe010fea0 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b0b7fe640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 0x7f0af80a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44702/0 (socket says 172.21.15.96:44702) 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b0bfff640 1 -- 172.21.15.96:0/502549858 learned_addr learned my addr 172.21.15.96:0/502549858 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.715+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe010f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe0112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b097fa640 1 -- 172.21.15.96:0/1807589183 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0afc0cee40 con 0x7f0b0c100700 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 -- 172.21.15.96:0/502549858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 msgr2=0x55bfe0112330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe0112330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 -- 172.21.15.96:0/502549858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 msgr2=0x55bfe010a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 -- 172.21.15.96:0/502549858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe0030460 con 0x55bfe010ad30 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b0b7fe640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 0x7f0af80a4d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0afc09b4b0 tx=0x7f0afc066a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe010f830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009bd0 tx=0x7f0af4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cbc818684e23b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b01ffb640 1 -- 172.21.15.96:0/502549858 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004230 con 0x55bfe010ad30 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b01ffb640 1 -- 172.21.15.96:0/502549858 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0af4009e40 con 0x55bfe010ad30 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.716+0000 7f0b01ffb640 1 -- 172.21.15.96:0/502549858 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004d40 con 0x55bfe010ad30 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- 172.21.15.96:0/502549858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 msgr2=0x55bfe010f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe010f830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009bd0 tx=0x7f0af4011810 comp rx=0 tx=0).stop 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- 172.21.15.96:0/502549858 shutdown_connections 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe010f830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/502549858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010a340 0x55bfe010a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- 172.21.15.96:0/502549858 >> 172.21.15.96:0/502549858 conn(0x55bfe01058b0 msgr2=0x55bfe0105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- 172.21.15.96:0/502549858 shutdown_connections 2024-03-18T15:47:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- 172.21.15.96:0/502549858 wait complete. 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.717+0000 7f0b14b8d740 1 -- start start 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 Processor -- start 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010a340 0x55bfe0114640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe011bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0112e50 con 0x55bfe010ad30 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bfe002bd70 con 0x55bfe010a340 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bfe0112fd0 con 0x55bfe010fea0 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b14b8d740 10 client.0 _ll_register_callbacks cb 0x55bfdfd83a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44708/0 (socket says 172.21.15.96:44708) 2024-03-18T15:47:27.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2817357907 learned_addr learned my addr 172.21.15.96:0/2817357907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010a340 0x55bfe0114640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe011bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 msgr2=0x55bfe011bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe011bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2817357907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010a340 msgr2=0x55bfe0114640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010a340 0x55bfe0114640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2817357907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfdfd84210 con 0x55bfe010ad30 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe011bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011d10 tx=0x7f0af40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b3c4238b589e9e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.718+0000 7f0aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4027020 con 0x55bfe010ad30 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0af4003c90 con 0x55bfe010ad30 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bfe007cf80 con 0x55bfe010ad30 2024-03-18T15:47:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0af4032050 con 0x55bfe010ad30 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 10 client.17352 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfdfe50620 con 0x55bfe010ad30 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bfe007cf80 con 0x55bfe010ad30 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f0af4018070 con 0x55bfe010ad30 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 10 client.17352 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 20 client.17352 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe0120220 0x55bfe0122670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 4 client.17352 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.719+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe00772d0 con 0x55bfe0120220 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.720+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe0120220 0x55bfe0122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.720+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe0120220 0x55bfe0122670 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30bd0dacd4d3527 server_cookie=24226030092c9d6 in_seq=0 out_seq=0 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.720+0000 7f0aeaffd640 10 client.17352 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2817357907 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bfe00772d0 con 0x55bfe0120220 2024-03-18T15:47:27.728 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0aeaffd640 10 client.17352 handle_command_reply: tid=0 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 client.17352 shutdown 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 msgr2=0x55bfe0114b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0af4011d10 tx=0x7f0af40038a0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 shutdown_connections 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010fea0 0x55bfe011bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe0120220 0x55bfe0122670 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0114b80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2817357907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010a340 0x55bfe0114640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0aeaffd640 0 client.17352 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 >> 172.21.15.96:0/2817357907 conn(0x55bfe01058b0 msgr2=0x55bfe0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 shutdown_connections 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2817357907 wait complete. 2024-03-18T15:47:27.729 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.721+0000 7f0b14b8d740 20 client.17352 trim_cache size 0 max 16384 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 Processor -- start 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 -- start start 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe0104320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe011bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1aaf20 con 0x55bfe010fea0 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1acaf0 con 0x55bfe010ad30 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ab620 con 0x55bfe010a340 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44716/0 (socket says 172.21.15.96:44716) 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 -- 172.21.15.96:0/2126428856 learned_addr learned my addr 172.21.15.96:0/2126428856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe011bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe0104320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 -- 172.21.15.96:0/2126428856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 msgr2=0x55bfe0104320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe0104320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 -- 172.21.15.96:0/2126428856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 msgr2=0x55bfe011bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe011bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 -- 172.21.15.96:0/2126428856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe00f7140 con 0x55bfe010fea0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005e30 tx=0x7f0b04032590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29172d153cc57484 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.736+0000 7f0b017fa640 1 -- 172.21.15.96:0/2126428856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04039040 con 0x55bfe010fea0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b017fa640 1 -- 172.21.15.96:0/2126428856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04003970 con 0x55bfe010fea0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b017fa640 1 -- 172.21.15.96:0/2126428856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0404c770 con 0x55bfe010fea0 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2126428856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 msgr2=0x55bfe0104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0b04005e30 tx=0x7f0b04032590 comp rx=0 tx=0).stop 2024-03-18T15:47:27.742 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2126428856 shutdown_connections 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010fea0 0x55bfe0104860 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010ad30 0x55bfe011bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2126428856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe0104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2126428856 >> 172.21.15.96:0/2126428856 conn(0x55bfe01058b0 msgr2=0x55bfe0100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2126428856 shutdown_connections 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2126428856 wait complete. 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.737+0000 7f0b14b8d740 1 -- start start 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 Processor -- start 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe01197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 0x55bfe011ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ac420 con 0x55bfe010ad30 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1abd20 con 0x55bfe010fea0 2024-03-18T15:47:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1a8f00 con 0x55bfe010a340 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b14b8d740 10 client.0 _ll_register_callbacks cb 0x7f0afc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44728/0 (socket says 172.21.15.96:44728) 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2757844477 learned_addr learned my addr 172.21.15.96:0/2757844477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe01197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 0x55bfe011ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 msgr2=0x55bfe01197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe01197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2757844477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 msgr2=0x55bfe011ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 0x55bfe011ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 -- 172.21.15.96:0/2757844477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe0112a40 con 0x55bfe010ad30 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 0x55bfe011ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0b02ffd640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0af4023f30 tx=0x7f0af4009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bba71650bfd968 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.738+0000 7f0aeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:27.744 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4027040 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0af4005450 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b04003970 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0af403d050 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 10 client.17358 fetch_fsmap learned FSMap version 41 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bfe01868a0 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04003970 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8378+0+0 (secure 0 0 0) 0x7f0af4018040 con 0x55bfe010ad30 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 10 client.17358 fetch_fsmap finished waiting for FSMap version 41 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 20 client.17358 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe01872a0 0x55bfe01896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 4 client.17358 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.739+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bfdfff5770 con 0x55bfe01872a0 2024-03-18T15:47:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.740+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe01872a0 0x55bfe01896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.740+0000 7f0b0bfff640 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe01872a0 0x55bfe01896c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fb46220d897d5b server_cookie=99d1ed1463026ec0 in_seq=0 out_seq=0 2024-03-18T15:47:27.746 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.740+0000 7f0aeaffd640 10 client.17358 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.749+0000 7f0aeaffd640 1 -- 172.21.15.96:0/2757844477 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bfdfff5770 con 0x55bfe01872a0 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.749+0000 7f0aeaffd640 10 client.17358 handle_command_reply: tid=0 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.749+0000 7f0b14b8d740 1 client.17358 shutdown 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 msgr2=0x55bfe0119cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0af4023f30 tx=0x7f0af4009b90 comp rx=0 tx=0).stop 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 shutdown_connections 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfe010fea0 0x55bfe011ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55bfe01872a0 0x55bfe01896c0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bfe010ad30 0x55bfe0119cf0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 --2- 172.21.15.96:0/2757844477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfe010a340 0x55bfe01197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0aeaffd640 0 client.17358 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 >> 172.21.15.96:0/2757844477 conn(0x55bfe01058b0 msgr2=0x55bfe010f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 shutdown_connections 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 1 -- 172.21.15.96:0/2757844477 wait complete. 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b14b8d740 20 client.17358 trim_cache size 0 max 16384 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:47:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:47:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 msgr2=0x7f0af80a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 0x7f0af80a4d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0afc09b4b0 tx=0x7f0afc066a10 comp rx=0 tx=0).stop 2024-03-18T15:47:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 msgr2=0x7f0b0c10af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.750+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00e9c0 tx=0x7f0afc00ee80 comp rx=0 tx=0).stop 2024-03-18T15:47:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 shutdown_connections 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0af80a28a0 0x7f0af80a4d60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b0c1022b0 0x7f0b0c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b0c1018f0 0x7f0b0c1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 --2- 172.21.15.96:0/1807589183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b0c100700 0x7f0b0c10af40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 >> 172.21.15.96:0/1807589183 conn(0x7f0b0c0fbc70 msgr2=0x7f0b0c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 shutdown_connections 2024-03-18T15:47:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:27.751+0000 7f0b11fa9640 1 -- 172.21.15.96:0/1807589183 wait complete. 2024-03-18T15:47:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:28 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:29.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:28 smithi137 ceph-mon[35868]: pgmap v574: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-03-18T15:47:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:28 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:29.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:28 smithi193 ceph-mon[36702]: pgmap v574: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-03-18T15:47:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:28 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:29.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:28 smithi096 ceph-mon[31930]: pgmap v574: 97 pgs: 97 active+clean; 493 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 529 op/s 2024-03-18T15:47:30.969 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:30 smithi193 ceph-mon[36702]: pgmap v575: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-03-18T15:47:31.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:30 smithi137 ceph-mon[35868]: pgmap v575: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-03-18T15:47:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:30 smithi096 ceph-mon[31930]: pgmap v575: 97 pgs: 97 active+clean; 495 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 577 op/s 2024-03-18T15:47:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:32 smithi137 ceph-mon[35868]: pgmap v576: 97 pgs: 97 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-18T15:47:33.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:32 smithi193 ceph-mon[36702]: pgmap v576: 97 pgs: 97 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-18T15:47:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:32 smithi096 ceph-mon[31930]: pgmap v576: 97 pgs: 97 active+clean; 480 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 281 op/s 2024-03-18T15:47:35.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:34 smithi137 ceph-mon[35868]: pgmap v577: 97 pgs: 97 active+clean; 465 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-03-18T15:47:35.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:34 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:35.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:34 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:35.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:34 smithi193 ceph-mon[36702]: pgmap v577: 97 pgs: 97 active+clean; 465 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-03-18T15:47:35.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:34 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:35.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:34 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:34 smithi096 ceph-mon[31930]: pgmap v577: 97 pgs: 97 active+clean; 465 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 275 op/s 2024-03-18T15:47:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:34 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:35.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:34 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:36.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:35 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:36.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:35 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:35 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:36.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:35 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:35 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:36.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:35 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:36 smithi137 ceph-mon[35868]: pgmap v578: 97 pgs: 97 active+clean; 456 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 259 op/s 2024-03-18T15:47:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:36 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:47:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:36 smithi193 ceph-mon[36702]: pgmap v578: 97 pgs: 97 active+clean; 456 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 259 op/s 2024-03-18T15:47:37.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:36 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:47:37.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:36 smithi096 ceph-mon[31930]: pgmap v578: 97 pgs: 97 active+clean; 456 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 259 op/s 2024-03-18T15:47:37.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:36 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:47:38.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:47:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:47:38.169 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-18T15:47:38.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:47:38.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:47:38.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:47:39.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:38 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-18T15:47:39.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:38 smithi137 ceph-mon[35868]: Unable to set osd_memory_target on smithi137 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T15:47:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:38 smithi137 ceph-mon[35868]: Adjusting osd_memory_target on smithi193 to 174.1M 2024-03-18T15:47:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:38 smithi137 ceph-mon[35868]: Unable to set osd_memory_target on smithi193 to 182638489: error parsing value: Value '182638489' is below minimum 939524096 2024-03-18T15:47:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:38 smithi137 ceph-mon[35868]: pgmap v579: 97 pgs: 97 active+clean; 455 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-03-18T15:47:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:38 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-18T15:47:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:38 smithi193 ceph-mon[36702]: Unable to set osd_memory_target on smithi137 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T15:47:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:38 smithi193 ceph-mon[36702]: Adjusting osd_memory_target on smithi193 to 174.1M 2024-03-18T15:47:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:38 smithi193 ceph-mon[36702]: Unable to set osd_memory_target on smithi193 to 182638489: error parsing value: Value '182638489' is below minimum 939524096 2024-03-18T15:47:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:38 smithi193 ceph-mon[36702]: pgmap v579: 97 pgs: 97 active+clean; 455 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-03-18T15:47:39.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:38 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi137 to 174.1M 2024-03-18T15:47:39.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:38 smithi096 ceph-mon[31930]: Unable to set osd_memory_target on smithi137 to 182618419: error parsing value: Value '182618419' is below minimum 939524096 2024-03-18T15:47:39.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:38 smithi096 ceph-mon[31930]: Adjusting osd_memory_target on smithi193 to 174.1M 2024-03-18T15:47:39.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:38 smithi096 ceph-mon[31930]: Unable to set osd_memory_target on smithi193 to 182638489: error parsing value: Value '182638489' is below minimum 939524096 2024-03-18T15:47:39.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:38 smithi096 ceph-mon[31930]: pgmap v579: 97 pgs: 97 active+clean; 455 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 218 op/s 2024-03-18T15:47:40.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:39 smithi137 ceph-mon[35868]: pgmap v580: 97 pgs: 97 active+clean; 444 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-03-18T15:47:40.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:39 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:39 smithi193 ceph-mon[36702]: pgmap v580: 97 pgs: 97 active+clean; 444 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-03-18T15:47:40.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:39 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:39 smithi096 ceph-mon[31930]: pgmap v580: 97 pgs: 97 active+clean; 444 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 331 op/s 2024-03-18T15:47:40.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:39 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 -- 172.21.15.96:0/2019907228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da0100700 msgr2=0x7f9da0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 --2- 172.21.15.96:0/2019907228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da0100700 0x7f9da0100b20 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009900 tx=0x7f9d9002bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 -- 172.21.15.96:0/2019907228 shutdown_connections 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 --2- 172.21.15.96:0/2019907228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da01022b0 0x7f9da0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 --2- 172.21.15.96:0/2019907228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01018f0 0x7f9da0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 --2- 172.21.15.96:0/2019907228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da0100700 0x7f9da0100b20 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 -- 172.21.15.96:0/2019907228 >> 172.21.15.96:0/2019907228 conn(0x7f9da00fbc70 msgr2=0x7f9da00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 -- 172.21.15.96:0/2019907228 shutdown_connections 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.839+0000 7f9da6026640 1 -- 172.21.15.96:0/2019907228 wait complete. 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 Processor -- start 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 -- start start 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da0100700 0x7f9da0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01022b0 0x7f9da019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010d4a0 con 0x7f9da01018f0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010d320 con 0x7f9da01022b0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9da6026640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da010d620 con 0x7f9da0100700 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57416/0 (socket says 172.21.15.96:57416) 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9d9effd640 1 -- 172.21.15.96:0/3232731471 learned_addr learned my addr 172.21.15.96:0/3232731471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da0100700 0x7f9da0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.840+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01022b0 0x7f9da019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 -- 172.21.15.96:0/3232731471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da0100700 msgr2=0x7f9da0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da0100700 0x7f9da0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 -- 172.21.15.96:0/3232731471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01022b0 msgr2=0x7f9da019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01022b0 0x7f9da019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 -- 172.21.15.96:0/3232731471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da019f690 con 0x7f9da01018f0 2024-03-18T15:47:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e9f0 tx=0x7f9d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9cff9640 1 -- 172.21.15.96:0/3232731471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400cd80 con 0x7f9da01018f0 2024-03-18T15:47:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da019f980 con 0x7f9da01018f0 2024-03-18T15:47:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9cff9640 1 -- 172.21.15.96:0/3232731471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94010910 con 0x7f9da01018f0 2024-03-18T15:47:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9d9cff9640 1 -- 172.21.15.96:0/3232731471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d940188f0 con 0x7f9da01018f0 2024-03-18T15:47:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.841+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da019ff20 con 0x7f9da01018f0 2024-03-18T15:47:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.843+0000 7f9d9cff9640 1 -- 172.21.15.96:0/3232731471 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f9d94010ac0 con 0x7f9da01018f0 2024-03-18T15:47:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 Processor -- start 2024-03-18T15:47:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9cff9640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 0x7f9d880a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 -- start start 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 0x7f9d880a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9cff9640 1 -- 172.21.15.96:0/3232731471 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9d940cee90 con 0x7f9da01018f0 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562146904a40 con 0x5621469a92b0 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5621468ad920 con 0x5621469a3750 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562146900ae0 con 0x5621469a4140 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57422/0 (socket says 172.21.15.96:57422) 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 -- 172.21.15.96:0/1813289276 learned_addr learned my addr 172.21.15.96:0/1813289276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9effd640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 0x7f9d880a4d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d98013830 tx=0x7f9d98013c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/1813289276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 msgr2=0x5621469a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/1813289276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 msgr2=0x5621469ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/1813289276 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621468c5460 con 0x5621469a3750 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9effd640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.844+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469a3b70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002c170 tx=0x7f9d9002c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c0ec3ff6ac67e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9d85ffb640 1 -- 172.21.15.96:0/1813289276 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9002cb80 con 0x5621469a3750 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9d85ffb640 1 -- 172.21.15.96:0/1813289276 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90005040 con 0x5621469a3750 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9d85ffb640 1 -- 172.21.15.96:0/1813289276 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004920 con 0x5621469a3750 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9da8c0b740 1 -- 172.21.15.96:0/1813289276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 msgr2=0x5621469a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.845+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469a3b70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002c170 tx=0x7f9d9002c660 comp rx=0 tx=0).stop 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- 172.21.15.96:0/1813289276 shutdown_connections 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/1813289276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469a3b70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- 172.21.15.96:0/1813289276 >> 172.21.15.96:0/1813289276 conn(0x56214699a8b0 msgr2=0x56214699acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- 172.21.15.96:0/1813289276 shutdown_connections 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- 172.21.15.96:0/1813289276 wait complete. 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- start start 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 Processor -- start 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a3750 0x5621469ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a4140 0x5621469adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621469ac200 con 0x5621469a92b0 2024-03-18T15:47:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5621468c0d70 con 0x5621469a4140 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621469ac380 con 0x5621469a3750 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9da8c0b740 10 client.0 _ll_register_callbacks cb 0x562146618a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57432/0 (socket says 172.21.15.96:57432) 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9d9ffff640 1 -- 172.21.15.96:0/2391368842 learned_addr learned my addr 172.21.15.96:0/2391368842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.846+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a3750 0x5621469ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9effd640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a4140 0x5621469adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 -- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a3750 msgr2=0x5621469ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a3750 0x5621469ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 -- 172.21.15.96:0/2391368842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a4140 msgr2=0x5621469adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a4140 0x5621469adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 -- 172.21.15.96:0/2391368842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562146619210 con 0x5621469a92b0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9d98003ba0 tx=0x7f9d98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451d3bb67002f3bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98004600 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9d980047a0 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90005040 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9d98010430 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9da8c0b740 10 client.17370 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5621466e5620 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.847+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90005040 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9d98031070 con 0x5621469a92b0 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 10 client.17370 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 20 client.17370 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5621469b9a30 0x5621469bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 4 client.17370 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562146911f80 con 0x5621469b9a30 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5621469b9a30 0x5621469bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9d7effd640 10 client.17370 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.848+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5621469b9a30 0x5621469bbea0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59463e0c400fb749 server_cookie=9ea044fddda459ad in_seq=0 out_seq=0 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9d7effd640 1 -- 172.21.15.96:0/2391368842 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562146911f80 con 0x5621469b9a30 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9d7effd640 10 client.17370 handle_command_reply: tid=0 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 client.17370 shutdown 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 msgr2=0x5621469b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9d98003ba0 tx=0x7f9d98004ab0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 shutdown_connections 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5621469b9a30 0x5621469bbea0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a92b0 0x5621469b4fd0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a4140 0x5621469adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2391368842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a3750 0x5621469ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 >> 172.21.15.96:0/2391368842 conn(0x56214699a8b0 msgr2=0x562146999bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9d7effd640 0 client.17370 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:40.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 shutdown_connections 2024-03-18T15:47:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2391368842 wait complete. 2024-03-18T15:47:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.849+0000 7f9da8c0b740 20 client.17370 trim_cache size 0 max 16384 2024-03-18T15:47:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 Processor -- start 2024-03-18T15:47:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 -- start start 2024-03-18T15:47:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a92b0 0x5621469b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a69f0 con 0x5621469a3750 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a85c0 con 0x5621469a92b0 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a70f0 con 0x5621469a4140 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57442/0 (socket says 172.21.15.96:57442) 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/2174313614 learned_addr learned my addr 172.21.15.96:0/2174313614 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9effd640 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a92b0 0x5621469b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/2174313614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 msgr2=0x5621469a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/2174313614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a92b0 msgr2=0x5621469b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.864+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a92b0 0x5621469b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9d9f7fe640 1 -- 172.21.15.96:0/2174313614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621469a3b30 con 0x5621469a3750 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x5621469a49a0 tx=0x7f9d90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa738532e29055c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9d857fa640 1 -- 172.21.15.96:0/2174313614 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90050b90 con 0x5621469a3750 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9d857fa640 1 -- 172.21.15.96:0/2174313614 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90005050 con 0x5621469a3750 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9d857fa640 1 -- 172.21.15.96:0/2174313614 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004920 con 0x5621469a3750 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2174313614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 msgr2=0x5621469a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x5621469a49a0 tx=0x7f9d90000bc0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2174313614 shutdown_connections 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a92b0 0x5621469b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a4140 0x5621469a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/2174313614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a3750 0x5621469a54a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.865+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2174313614 >> 172.21.15.96:0/2174313614 conn(0x56214699a8b0 msgr2=0x562146995a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2174313614 shutdown_connections 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- 172.21.15.96:0/2174313614 wait complete. 2024-03-18T15:47:40.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- start start 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 Processor -- start 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a92b0 0x5621469b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a7ef0 con 0x5621469a4140 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a77f0 con 0x5621469a3750 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9da01a49d0 con 0x5621469a92b0 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9da8c0b740 10 client.0 _ll_register_callbacks cb 0x7f9d9800fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9d9effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57446/0 (socket says 172.21.15.96:57446) 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9d9effd640 1 -- 172.21.15.96:0/3372150323 learned_addr learned my addr 172.21.15.96:0/3372150323 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.866+0000 7f9d9ffff640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a92b0 0x5621469b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 -- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a92b0 msgr2=0x5621469b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a92b0 0x5621469b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 -- 172.21.15.96:0/3372150323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 msgr2=0x5621469b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 -- 172.21.15.96:0/3372150323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621469b3fb0 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d9effd640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409b550 tx=0x7f9d94086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dca04361cbf8d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d940b9060 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f9d940afd80 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90005050 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9d9409aca0 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 10 client.17376 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5621469b53c0 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90005050 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f9d9409cad0 con 0x5621469a4140 2024-03-18T15:47:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 20 client.17376 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562146a20610 0x562146a22a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 4 client.17376 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.867+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56214690c2d0 con 0x562146a20610 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.868+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562146a20610 0x562146a22a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.868+0000 7f9d7effd640 10 client.17376 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:40.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.868+0000 7f9d9f7fe640 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562146a20610 0x562146a22a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524d4edad9b43753 server_cookie=25d35e661874f9 in_seq=0 out_seq=0 2024-03-18T15:47:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9d7effd640 1 -- 172.21.15.96:0/3372150323 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56214690c2d0 con 0x562146a20610 2024-03-18T15:47:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9d7effd640 10 client.17376 handle_command_reply: tid=0 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 client.17376 shutdown 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 msgr2=0x5621469b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9d9409b550 tx=0x7f9d94086040 comp rx=0 tx=0).stop 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 shutdown_connections 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621469a92b0 0x5621469b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562146a20610 0x562146a22a30 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621469a4140 0x5621469b29b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 --2- 172.21.15.96:0/3372150323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621469a3750 0x5621469b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9d7effd640 0 client.17376 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 >> 172.21.15.96:0/3372150323 conn(0x56214699a8b0 msgr2=0x5621469ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 shutdown_connections 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 1 -- 172.21.15.96:0/3372150323 wait complete. 2024-03-18T15:47:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.878+0000 7f9da8c0b740 20 client.17376 trim_cache size 0 max 16384 2024-03-18T15:47:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:47:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:47:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:47:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 msgr2=0x7f9d880a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 0x7f9d880a4d60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9d98013830 tx=0x7f9d98013c30 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 msgr2=0x7f9da0197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400e9f0 tx=0x7f9d9400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 shutdown_connections 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f9d880a28a0 0x7f9d880a4d60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9da01022b0 0x7f9da019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9da01018f0 0x7f9da0197fb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 --2- 172.21.15.96:0/3232731471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9da0100700 0x7f9da0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 >> 172.21.15.96:0/3232731471 conn(0x7f9da00fbc70 msgr2=0x7f9da0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 shutdown_connections 2024-03-18T15:47:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:40.879+0000 7f9da6026640 1 -- 172.21.15.96:0/3232731471 wait complete. 2024-03-18T15:47:41.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:47:41.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/4093889758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e01018d0 msgr2=0x7fb4e0101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/4093889758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e01018d0 0x7fb4e0101d50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009900 tx=0x7fb4cc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/4093889758 shutdown_connections 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/4093889758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e0102290 0x7fb4e0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/4093889758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e01018d0 0x7fb4e0101d50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/4093889758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01006e0 0x7fb4e0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/4093889758 >> 172.21.15.96:0/4093889758 conn(0x7fb4e00fbc70 msgr2=0x7fb4e00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/4093889758 shutdown_connections 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/4093889758 wait complete. 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 Processor -- start 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.625+0000 7fb4e6c6c640 1 -- start start 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e01006e0 0x7fb4e0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01018d0 0x7fb4e0196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010c340 con 0x7fb4e0102290 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010c1c0 con 0x7fb4e01018d0 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010c4c0 con 0x7fb4e01006e0 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57466/0 (socket says 172.21.15.96:57466) 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e4a08640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e01006e0 0x7fb4e0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 -- 172.21.15.96:0/720049888 learned_addr learned my addr 172.21.15.96:0/720049888 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4dffff640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01018d0 0x7fb4e0196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 -- 172.21.15.96:0/720049888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e01006e0 msgr2=0x7fb4e0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e01006e0 0x7fb4e0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 -- 172.21.15.96:0/720049888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01018d0 msgr2=0x7fb4e0196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01018d0 0x7fb4e0196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 -- 172.21.15.96:0/720049888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e019e630 con 0x7fb4e0102290 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e5209640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800eaa0 tx=0x7fb4d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4ddffb640 1 -- 172.21.15.96:0/720049888 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800cdb0 con 0x7fb4e0102290 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4ddffb640 1 -- 172.21.15.96:0/720049888 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8005020 con 0x7fb4e0102290 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e019e920 con 0x7fb4e0102290 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.626+0000 7fb4ddffb640 1 -- 172.21.15.96:0/720049888 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80116b0 con 0x7fb4e0102290 2024-03-18T15:47:41.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.627+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e019edf0 con 0x7fb4e0102290 2024-03-18T15:47:41.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.628+0000 7fb4ddffb640 1 -- 172.21.15.96:0/720049888 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb4d8005560 con 0x7fb4e0102290 2024-03-18T15:47:41.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 Processor -- start 2024-03-18T15:47:41.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4ddffb640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 0x7fb4d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 -- start start 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4ddffb640 1 -- 172.21.15.96:0/720049888 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb4d80cebf0 con 0x7fb4e0102290 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4dffff640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 0x7fb4d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x559288703590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x559288708660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x55928870b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559288668500 con 0x559288703b60 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559288610be0 con 0x559288703170 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e9858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592886645a0 con 0x559288708cd0 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x559288708660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e4a08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x559288703590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x559288708660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57468/0 (socket says 172.21.15.96:57468) 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4dffff640 1 -- 172.21.15.96:0/582017568 learned_addr learned my addr 172.21.15.96:0/582017568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.629+0000 7fb4e5209640 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x55928870b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 -- 172.21.15.96:0/582017568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 msgr2=0x55928870b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x55928870b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 -- 172.21.15.96:0/582017568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 msgr2=0x559288708660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4dffff640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 0x7fb4d00a4cc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0096c0 tx=0x7fb4cc02cb90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x559288708660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 -- 172.21.15.96:0/582017568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592886244b0 con 0x559288703170 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x559288703590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0101730 tx=0x7fb4d400bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3885ab82bda2d20 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4caffd640 1 -- 172.21.15.96:0/582017568 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4003d50 con 0x559288703170 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4caffd640 1 -- 172.21.15.96:0/582017568 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4015070 con 0x559288703170 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.630+0000 7fb4caffd640 1 -- 172.21.15.96:0/582017568 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4011680 con 0x559288703170 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- 172.21.15.96:0/582017568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 msgr2=0x559288703590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x559288703590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0101730 tx=0x7fb4d400bbe0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- 172.21.15.96:0/582017568 shutdown_connections 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x55928870b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x559288708660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- 172.21.15.96:0/582017568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x559288703590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- 172.21.15.96:0/582017568 >> 172.21.15.96:0/582017568 conn(0x5592886fe6e0 msgr2=0x5592886feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- 172.21.15.96:0/582017568 shutdown_connections 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- 172.21.15.96:0/582017568 wait complete. 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- start start 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 Processor -- start 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x55928870d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x559288714c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55928870bc80 con 0x559288703b60 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559288624160 con 0x559288703170 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.631+0000 7fb4e9858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55928870be00 con 0x559288708cd0 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e9858740 10 client.0 _ll_register_callbacks cb 0x559288608150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57480/0 (socket says 172.21.15.96:57480) 2024-03-18T15:47:41.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 -- 172.21.15.96:0/887630485 learned_addr learned my addr 172.21.15.96:0/887630485 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e5209640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x559288714c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x55928870d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 -- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 msgr2=0x559288714c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x559288714c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 -- 172.21.15.96:0/887630485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 msgr2=0x55928870d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x55928870d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 -- 172.21.15.96:0/887630485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592884467d0 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4dffff640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009230 tx=0x7fb4cc044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858879cbefc8ed7c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4bffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0055d0 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb4cc0046c0 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d4015070 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb4cc0048c0 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e9858740 10 client.17388 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.632+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55928837c3f0 con 0x559288703b60 2024-03-18T15:47:41.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4015070 con 0x559288703b60 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fb4cc05d070 con 0x559288703b60 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 10 client.17388 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 20 client.17388 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5592887191a0 0x55928871b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 4 client.17388 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559288675a40 con 0x5592887191a0 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5592887191a0 0x55928871b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4bffff640 10 client.17388 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.633+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5592887191a0 0x55928871b640 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a651936dd2085df1 server_cookie=52bdbde60306f1f4 in_seq=0 out_seq=0 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4bffff640 1 -- 172.21.15.96:0/887630485 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559288675a40 con 0x5592887191a0 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4bffff640 10 client.17388 handle_command_reply: tid=0 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 client.17388 shutdown 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 msgr2=0x55928870dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009230 tx=0x7fb4cc044040 comp rx=0 tx=0).stop 2024-03-18T15:47:41.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 shutdown_connections 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288708cd0 0x559288714c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5592887191a0 0x55928871b640 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x55928870dc30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4bffff640 0 client.17388 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 --2- 172.21.15.96:0/887630485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703170 0x55928870d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 >> 172.21.15.96:0/887630485 conn(0x5592886fe6e0 msgr2=0x5592886fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 shutdown_connections 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 1 -- 172.21.15.96:0/887630485 wait complete. 2024-03-18T15:47:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.634+0000 7fb4e9858740 20 client.17388 trim_cache size 0 max 16384 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 Processor -- start 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 -- start start 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592886fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 0x5592886fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a7ce0 con 0x559288708cd0 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a98b0 con 0x559288703b60 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e9858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a83e0 con 0x559288703170 2024-03-18T15:47:41.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57484/0 (socket says 172.21.15.96:57484) 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 -- 172.21.15.96:0/2260238120 learned_addr learned my addr 172.21.15.96:0/2260238120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4dffff640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 0x5592886fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592886fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 -- 172.21.15.96:0/2260238120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 msgr2=0x5592886fd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592886fd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 -- 172.21.15.96:0/2260238120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 msgr2=0x5592886fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 0x5592886fd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.652+0000 7fb4e5209640 1 -- 172.21.15.96:0/2260238120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928833c0f0 con 0x559288708cd0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4dffff640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 0x5592886fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4e5209640 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80a37d0 tx=0x7fb4d8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56d44d6fbd34626 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4ca7fc640 1 -- 172.21.15.96:0/2260238120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d801f040 con 0x559288708cd0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4ca7fc640 1 -- 172.21.15.96:0/2260238120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8097040 con 0x559288708cd0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4ca7fc640 1 -- 172.21.15.96:0/2260238120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8084e00 con 0x559288708cd0 2024-03-18T15:47:41.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4e9858740 1 -- 172.21.15.96:0/2260238120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 msgr2=0x5592886fdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.653+0000 7fb4e9858740 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80a37d0 tx=0x7fb4d8086040 comp rx=0 tx=0).stop 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- 172.21.15.96:0/2260238120 shutdown_connections 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288708cd0 0x5592886fdaf0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288703b60 0x5592886fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- 172.21.15.96:0/2260238120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592886fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- 172.21.15.96:0/2260238120 >> 172.21.15.96:0/2260238120 conn(0x5592886fe6e0 msgr2=0x559288704570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- 172.21.15.96:0/2260238120 shutdown_connections 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- 172.21.15.96:0/2260238120 wait complete. 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- start start 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 Processor -- start 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592887142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 0x55928877fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a91e0 con 0x559288703b60 2024-03-18T15:47:41.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a8ae0 con 0x559288708cd0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e01a5cc0 con 0x559288703170 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e9858740 10 client.0 _ll_register_callbacks cb 0x7fb4cc0044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57486/0 (socket says 172.21.15.96:57486) 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4dffff640 1 -- 172.21.15.96:0/1215510840 learned_addr learned my addr 172.21.15.96:0/1215510840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592887142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.654+0000 7fb4e5209640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 0x55928877fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 -- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 msgr2=0x5592887142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592887142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 -- 172.21.15.96:0/1215510840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 msgr2=0x55928877fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 0x55928877fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 -- 172.21.15.96:0/1215510840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928862f9d0 con 0x559288703b60 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4e5209640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 0x55928877fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4dffff640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc003d70 tx=0x7fb4cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e65879dfdcb9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc004a60 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb4cc004c00 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d8097040 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fb4cc004e00 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4e9858740 10 client.17394 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559288781240 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d8097040 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.655+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fb4cc05d040 con 0x559288703b60 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 10 client.17394 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 20 client.17394 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559288781c90 0x5592887840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 4 client.17394 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55928866fd90 con 0x559288781c90 2024-03-18T15:47:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559288781c90 0x5592887840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e4a08640 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559288781c90 0x5592887840b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9504861ce3f1ffc8 server_cookie=ddcae3106eda8843 in_seq=0 out_seq=0 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4bffff640 10 client.17394 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4bffff640 1 -- 172.21.15.96:0/1215510840 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55928866fd90 con 0x559288781c90 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4bffff640 10 client.17394 handle_command_reply: tid=0 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 client.17394 shutdown 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 msgr2=0x5592887147f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc003d70 tx=0x7fb4cc005fd0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 shutdown_connections 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559288708cd0 0x55928877fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x559288781c90 0x5592887840b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559288703b60 0x5592887147f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 --2- 172.21.15.96:0/1215510840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559288703170 0x5592887142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4bffff640 0 client.17394 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.656+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 >> 172.21.15.96:0/1215510840 conn(0x5592886fe6e0 msgr2=0x559288701940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 shutdown_connections 2024-03-18T15:47:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e9858740 1 -- 172.21.15.96:0/1215510840 wait complete. 2024-03-18T15:47:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e9858740 20 client.17394 trim_cache size 0 max 16384 2024-03-18T15:47:41.664 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:47:41.664 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:47:41.664 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 msgr2=0x7fb4d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 0x7fb4d00a4cc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0096c0 tx=0x7fb4cc02cb90 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 msgr2=0x7fb4e019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800eaa0 tx=0x7fb4d800ef60 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 shutdown_connections 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb4d00a2800 0x7fb4d00a4cc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4e0102290 0x7fb4e019dfc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4e01018d0 0x7fb4e0196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 --2- 172.21.15.96:0/720049888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4e01006e0 0x7fb4e0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 >> 172.21.15.96:0/720049888 conn(0x7fb4e00fbc70 msgr2=0x7fb4e0104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 shutdown_connections 2024-03-18T15:47:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.657+0000 7fb4e6c6c640 1 -- 172.21.15.96:0/720049888 wait complete. 2024-03-18T15:47:41.670 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:47:41.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.788+0000 7fc70ad3b640 1 -- 172.21.15.96:0/4243327976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc704100700 msgr2=0x7fc704100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/4243327976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc704100700 0x7fc704100b20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc6f8009900 tx=0x7fc6f802bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 -- 172.21.15.96:0/4243327976 shutdown_connections 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/4243327976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041022b0 0x7fc704108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/4243327976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7041018f0 0x7fc704101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/4243327976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc704100700 0x7fc704100b20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 -- 172.21.15.96:0/4243327976 >> 172.21.15.96:0/4243327976 conn(0x7fc7040fbc70 msgr2=0x7fc7040fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 -- 172.21.15.96:0/4243327976 shutdown_connections 2024-03-18T15:47:41.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.789+0000 7fc70ad3b640 1 -- 172.21.15.96:0/4243327976 wait complete. 2024-03-18T15:47:41.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 Processor -- start 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 -- start start 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc704100700 0x7fc70410b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410d4c0 con 0x7fc7041022b0 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410d340 con 0x7fc7041018f0 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc70ad3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410d640 con 0x7fc704100700 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc703fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc7092d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc703fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:48244/0 (socket says 172.21.15.96:48244) 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc7092d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57500/0 (socket says 172.21.15.96:57500) 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc703fff640 1 -- 172.21.15.96:0/896390512 learned_addr learned my addr 172.21.15.96:0/896390512 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.790+0000 7fc708ad7640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc704100700 0x7fc70410b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 -- 172.21.15.96:0/896390512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc704100700 msgr2=0x7fc70410b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc704100700 0x7fc70410b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 -- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 msgr2=0x7fc704109c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 -- 172.21.15.96:0/896390512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70410a490 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc7092d8640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc703fff640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400cd80 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4010910 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7041a1c30 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.791+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4018990 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.792+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a21d0 con 0x7fc7041018f0 2024-03-18T15:47:41.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.792+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc704107a20 con 0x7fc7041018f0 2024-03-18T15:47:41.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.794+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc6f4010ac0 con 0x7fc7041018f0 2024-03-18T15:47:41.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.795+0000 7fc701ffb640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:41.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.795+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc6f40cf270 con 0x7fc7041018f0 2024-03-18T15:47:41.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.795+0000 7fc708ad7640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:41.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.795+0000 7fc708ad7640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6f802c170 tx=0x7fc6f802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:41.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:41.802+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc6f409c2e0 con 0x7fc7041018f0 2024-03-18T15:47:42.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.040+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc70402d3d0 con 0x7fc7041018f0 2024-03-18T15:47:42.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.041+0000 7fc701ffb640 1 -- 172.21.15.96:0/896390512 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11940 (secure 0 0 0) 0x7fc6f409bbe0 con 0x7fc7041018f0 2024-03-18T15:47:42.042 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:47:42.042 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:47:38.752260+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24452,24460,24485,24493,24472]},"id":1}]} 2024-03-18T15:47:42.043 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 42 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 msgr2=0x7fc6ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc6f802c170 tx=0x7fc6f802df90 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 msgr2=0x7fc7041096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e9f0 tx=0x7fc6f400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 shutdown_connections 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc6ec0a2880 0x7fc6ec0a4d40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7041022b0 0x7fc704109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7041018f0 0x7fc7041096c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 --2- 172.21.15.96:0/896390512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc704100700 0x7fc70410b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.043+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 >> 172.21.15.96:0/896390512 conn(0x7fc7040fbc70 msgr2=0x7fc704104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.044+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 shutdown_connections 2024-03-18T15:47:42.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.044+0000 7fc70ad3b640 1 -- 172.21.15.96:0/896390512 wait complete. 2024-03-18T15:47:42.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 -- 172.21.15.96:0/422999955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 msgr2=0x7fccc0101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 --2- 172.21.15.96:0/422999955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0101d50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009900 tx=0x7fccb402bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 -- 172.21.15.96:0/422999955 shutdown_connections 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 --2- 172.21.15.96:0/422999955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc0108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 --2- 172.21.15.96:0/422999955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0101d50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 --2- 172.21.15.96:0/422999955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 0x7fccc0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 -- 172.21.15.96:0/422999955 >> 172.21.15.96:0/422999955 conn(0x7fccc00fbc70 msgr2=0x7fccc00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 -- 172.21.15.96:0/422999955 shutdown_connections 2024-03-18T15:47:42.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 -- 172.21.15.96:0/422999955 wait complete. 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.161+0000 7fccc5d77640 1 Processor -- start 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 -- start start 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 0x7fccc0196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc019dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010c360 con 0x7fccc01018d0 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010c1e0 con 0x7fccc0102290 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccc5d77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccc010c4e0 con 0x7fccc01006e0 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57522/0 (socket says 172.21.15.96:57522) 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 -- 172.21.15.96:0/3190756251 learned_addr learned my addr 172.21.15.96:0/3190756251 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbffff640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc019dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 0x7fccc0196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 -- 172.21.15.96:0/3190756251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 msgr2=0x7fccc0196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 0x7fccc0196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 -- 172.21.15.96:0/3190756251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 msgr2=0x7fccc019dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc019dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 -- 172.21.15.96:0/3190756251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc019e650 con 0x7fccc01018d0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbffff640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.162+0000 7fccbeffd640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009230 tx=0x7fccb402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.163+0000 7fccbcff9640 1 -- 172.21.15.96:0/3190756251 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4004450 con 0x7fccc01018d0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.163+0000 7fccbcff9640 1 -- 172.21.15.96:0/3190756251 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb40054e0 con 0x7fccc01018d0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.163+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc0068940 con 0x7fccc01018d0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.163+0000 7fccbcff9640 1 -- 172.21.15.96:0/3190756251 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb403b440 con 0x7fccc01018d0 2024-03-18T15:47:42.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.163+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc0068e30 con 0x7fccc01018d0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.165+0000 7fccbcff9640 1 -- 172.21.15.96:0/3190756251 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fccb40035f0 con 0x7fccc01018d0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.165+0000 7fccc8964740 1 Processor -- start 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 -- start start 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbe9f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561dbe958490 con 0x561dbe9f93b0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dbe900c90 con 0x561dbe9f4240 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccc8964740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561dbe954530 con 0x561dbe9f3850 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbe9f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:50270/0 (socket says 172.21.15.96:50270) 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbffff640 1 -- 172.21.15.96:0/3419726899 learned_addr learned my addr 172.21.15.96:0/3419726899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbcff9640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 0x7fcca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 -- 172.21.15.96:0/3419726899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 msgr2=0x561dbe9f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 -- 172.21.15.96:0/3419726899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 msgr2=0x561dbe9fb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 -- 172.21.15.96:0/3419726899 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbe9bb500 con 0x561dbe9f4240 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbffff640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 0x7fcca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbcff9640 1 -- 172.21.15.96:0/3190756251 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fccb40f2630 con 0x7fccc01018d0 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbeffd640 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.166+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbe9f8d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009bd0 tx=0x7fccb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe95e762ccc062e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fcca5ffb640 1 -- 172.21.15.96:0/3419726899 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0004230 con 0x561dbe9f4240 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccbffff640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 0x7fcca80a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x561dbe9f4aa0 tx=0x7fccb8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fcca5ffb640 1 -- 172.21.15.96:0/3419726899 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb0009e40 con 0x561dbe9f4240 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fcca5ffb640 1 -- 172.21.15.96:0/3419726899 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00197c0 con 0x561dbe9f4240 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 -- 172.21.15.96:0/3419726899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 msgr2=0x561dbe9f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbe9f8d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009bd0 tx=0x7fccb0011810 comp rx=0 tx=0).stop 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 -- 172.21.15.96:0/3419726899 shutdown_connections 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbe9f8d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 --2- 172.21.15.96:0/3419726899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 -- 172.21.15.96:0/3419726899 >> 172.21.15.96:0/3419726899 conn(0x561dbe9eedc0 msgr2=0x561dbe9ef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 -- 172.21.15.96:0/3419726899 shutdown_connections 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.167+0000 7fccc8964740 1 -- 172.21.15.96:0/3419726899 wait complete. 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 -- start start 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 Processor -- start 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f4240 0x561dbe9fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 0x561dbea052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561dbe9fc360 con 0x561dbe9f3850 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561dbe914230 con 0x561dbe9f93b0 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561dbe9fc4e0 con 0x561dbe9f4240 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccc8964740 10 client.0 _ll_register_callbacks cb 0x561dbe950010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 0x561dbea052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57538/0 (socket says 172.21.15.96:57538) 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 -- 172.21.15.96:0/1487788867 learned_addr learned my addr 172.21.15.96:0/1487788867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbeffd640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f4240 0x561dbe9fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 -- 172.21.15.96:0/1487788867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f4240 msgr2=0x561dbe9fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f4240 0x561dbe9fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 -- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 msgr2=0x561dbea052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 0x561dbea052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbf7fe640 1 -- 172.21.15.96:0/1487788867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbe923100 con 0x561dbe9f3850 2024-03-18T15:47:42.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.168+0000 7fccbffff640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 0x561dbea052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fccb0005890 tx=0x7fccb0009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1bd22eb29a430d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0027070 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fccb00054d0 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561dbe9659d0 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fccb0033050 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 10 client.17409 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561dbe66c3f0 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561dbe9659d0 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fccb00234b0 con 0x561dbe9f3850 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 10 client.17409 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 10 client.17409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 20 client.17409 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea09930 0x561dbea0bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 4 client.17409 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:42.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.169+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dbe95fd20 con 0x561dbea09930 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.170+0000 7fccbeffd640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea09930 0x561dbea0bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.170+0000 7fcc9effd640 10 client.17409 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.170+0000 7fccbeffd640 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea09930 0x561dbea0bd70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=beb9101954be84b2 server_cookie=b5281bd9e3db6207 in_seq=0 out_seq=0 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fcc9effd640 1 -- 172.21.15.96:0/1487788867 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561dbe95fd20 con 0x561dbea09930 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fcc9effd640 10 client.17409 handle_command_reply: tid=0 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 client.17409 shutdown 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 msgr2=0x561dbe9fdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fccb0005890 tx=0x7fccb0009830 comp rx=0 tx=0).stop 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 shutdown_connections 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea09930 0x561dbea0bd70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f93b0 0x561dbea052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fcc9effd640 0 client.17409 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f4240 0x561dbe9fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 --2- 172.21.15.96:0/1487788867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f3850 0x561dbe9fdda0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 >> 172.21.15.96:0/1487788867 conn(0x561dbe9eedc0 msgr2=0x561dbe9edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 shutdown_connections 2024-03-18T15:47:42.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 1 -- 172.21.15.96:0/1487788867 wait complete. 2024-03-18T15:47:42.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.171+0000 7fccc8964740 20 client.17409 trim_cache size 0 max 16384 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 Processor -- start 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 -- start start 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a6b40 con 0x561dbe9f93b0 2024-03-18T15:47:42.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a8710 con 0x561dbe9f4240 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccc8964740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a7240 con 0x561dbe9f3850 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccbffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccbffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57540/0 (socket says 172.21.15.96:57540) 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.184+0000 7fccbffff640 1 -- 172.21.15.96:0/1876124629 learned_addr learned my addr 172.21.15.96:0/1876124629 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 -- 172.21.15.96:0/1876124629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 msgr2=0x561dbe9f5a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 -- 172.21.15.96:0/1876124629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 msgr2=0x561dbea05270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea05270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 -- 172.21.15.96:0/1876124629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbe7367d0 con 0x561dbe9f93b0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fccbffff640 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fccb801c660 tx=0x7fccb801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198636ca8ed5d1de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fcca57fa640 1 -- 172.21.15.96:0/1876124629 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb802d070 con 0x561dbe9f93b0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fcca57fa640 1 -- 172.21.15.96:0/1876124629 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb8003e50 con 0x561dbe9f93b0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.185+0000 7fcca57fa640 1 -- 172.21.15.96:0/1876124629 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8004820 con 0x561dbe9f93b0 2024-03-18T15:47:42.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- 172.21.15.96:0/1876124629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 msgr2=0x561dbe9fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fccb801c660 tx=0x7fccb801a000 comp rx=0 tx=0).stop 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- 172.21.15.96:0/1876124629 shutdown_connections 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbe9fd3c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- 172.21.15.96:0/1876124629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbe9f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- 172.21.15.96:0/1876124629 >> 172.21.15.96:0/1876124629 conn(0x561dbe9eedc0 msgr2=0x561dbe9edfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- 172.21.15.96:0/1876124629 shutdown_connections 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- 172.21.15.96:0/1876124629 wait complete. 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- start start 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 Processor -- start 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbea029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea02ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a8040 con 0x561dbe9f93b0 2024-03-18T15:47:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a7940 con 0x561dbe9f4240 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fccc01a4b20 con 0x561dbe9f3850 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccc8964740 10 client.0 _ll_register_callbacks cb 0x7fccb40f8420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccbffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccbffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57544/0 (socket says 172.21.15.96:57544) 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccbffff640 1 -- 172.21.15.96:0/1659621080 learned_addr learned my addr 172.21.15.96:0/1659621080 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.186+0000 7fccbeffd640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea02ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 -- 172.21.15.96:0/1659621080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 msgr2=0x561dbea029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbea029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 -- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 msgr2=0x561dbea02ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea02ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 -- 172.21.15.96:0/1659621080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbea044e0 con 0x561dbe9f93b0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccbffff640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fccb8004af0 tx=0x7fccb8011320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddc6679aa28dd35 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb802d040 con 0x561dbe9f93b0 2024-03-18T15:47:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fccb8003970 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccb8003e50 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fccb8003b70 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccc8964740 10 client.17415 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561dbea6fc70 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccb8003e50 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.187+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fccb803a030 con 0x561dbe9f93b0 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 10 client.17415 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 10 client.17415 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 20 client.17415 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea70610 0x561dbea72a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 4 client.17415 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.188+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dbe8dd6a0 con 0x561dbea70610 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.189+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea70610 0x561dbea72a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.192+0000 7fccbf7fe640 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea70610 0x561dbea72a30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3fd325bfff5aee44 server_cookie=b64cd00900ab85c9 in_seq=0 out_seq=0 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.192+0000 7fcc9effd640 10 client.17415 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.192+0000 7fcc9effd640 1 -- 172.21.15.96:0/1659621080 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dbe8dd6a0 con 0x561dbea70610 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.192+0000 7fcc9effd640 10 client.17415 handle_command_reply: tid=0 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 client.17415 shutdown 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 msgr2=0x561dbea03ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fccb8004af0 tx=0x7fccb8011320 comp rx=0 tx=0).stop 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 shutdown_connections 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x561dbea70610 0x561dbea72a30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dbe9f93b0 0x561dbea03ef0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561dbe9f4240 0x561dbea02ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 --2- 172.21.15.96:0/1659621080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dbe9f3850 0x561dbea029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fcc9effd640 0 client.17415 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:47:42.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 >> 172.21.15.96:0/1659621080 conn(0x561dbe9eedc0 msgr2=0x561dbe9ed750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 shutdown_connections 2024-03-18T15:47:42.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 1 -- 172.21.15.96:0/1659621080 wait complete. 2024-03-18T15:47:42.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc8964740 20 client.17415 trim_cache size 0 max 16384 2024-03-18T15:47:42.196 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 msgr2=0x7fcca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 0x7fcca80a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x561dbe9f4aa0 tx=0x7fccb8004a40 comp rx=0 tx=0).stop 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 msgr2=0x7fccc0196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009230 tx=0x7fccb402c650 comp rx=0 tx=0).stop 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 shutdown_connections 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fcca80a2800 0x7fcca80a4cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fccc0102290 0x7fccc019dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccc01018d0 0x7fccc0196f70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 --2- 172.21.15.96:0/3190756251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccc01006e0 0x7fccc0196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.193+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 >> 172.21.15.96:0/3190756251 conn(0x7fccc00fbc70 msgr2=0x7fccc0104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.194+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 shutdown_connections 2024-03-18T15:47:42.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.194+0000 7fccc5d77640 1 -- 172.21.15.96:0/3190756251 wait complete. 2024-03-18T15:47:42.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- 172.21.15.96:0/1520698080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41006e0 msgr2=0x7fa1f4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 --2- 172.21.15.96:0/1520698080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41006e0 0x7fa1f4100b00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009900 tx=0x7fa1e802bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- 172.21.15.96:0/1520698080 shutdown_connections 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 --2- 172.21.15.96:0/1520698080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 0x7fa1f4108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 --2- 172.21.15.96:0/1520698080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41018d0 0x7fa1f4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 --2- 172.21.15.96:0/1520698080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41006e0 0x7fa1f4100b00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- 172.21.15.96:0/1520698080 >> 172.21.15.96:0/1520698080 conn(0x7fa1f40fbc70 msgr2=0x7fa1f40fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- 172.21.15.96:0/1520698080 shutdown_connections 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- 172.21.15.96:0/1520698080 wait complete. 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 Processor -- start 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.310+0000 7fa1fa390640 1 -- start start 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41006e0 0x7fa1f4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 0x7fa1f419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410c350 con 0x7fa1f41018d0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410c1d0 con 0x7fa1f4102290 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f410c4d0 con 0x7fa1f41006e0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57564/0 (socket says 172.21.15.96:57564) 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 -- 172.21.15.96:0/308626030 learned_addr learned my addr 172.21.15.96:0/308626030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41006e0 0x7fa1f4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f892d640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 0x7fa1f419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 -- 172.21.15.96:0/308626030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41006e0 msgr2=0x7fa1f4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41006e0 0x7fa1f4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 -- 172.21.15.96:0/308626030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 msgr2=0x7fa1f419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 0x7fa1f419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 -- 172.21.15.96:0/308626030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f419e670 con 0x7fa1f41018d0 2024-03-18T15:47:42.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9f0 tx=0x7fa1e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f17fa640 1 -- 172.21.15.96:0/308626030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400cd80 con 0x7fa1f41018d0 2024-03-18T15:47:42.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f419e960 con 0x7fa1f41018d0 2024-03-18T15:47:42.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f17fa640 1 -- 172.21.15.96:0/308626030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e4005500 con 0x7fa1f41018d0 2024-03-18T15:47:42.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1f17fa640 1 -- 172.21.15.96:0/308626030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4010690 con 0x7fa1f41018d0 2024-03-18T15:47:42.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.311+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f419ee60 con 0x7fa1f41018d0 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.313+0000 7fa1f17fa640 1 -- 172.21.15.96:0/308626030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fa1e40046c0 con 0x7fa1f41018d0 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f17fa640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 0x7fa1dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 Processor -- start 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 -- start start 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 0x7fa1dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f17fa640 1 -- 172.21.15.96:0/308626030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa1e40cee70 con 0x7fa1f41018d0 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a64423490 con 0x559a644be850 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a643cbc90 con 0x559a644c43b0 2024-03-18T15:47:42.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a6441f530 con 0x559a644bf240 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f892d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57566/0 (socket says 172.21.15.96:57566) 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.314+0000 7fa1f3fff640 1 -- 172.21.15.96:0/2984584903 learned_addr learned my addr 172.21.15.96:0/2984584903 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 0x7fa1dc0a4cc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80096c0 tx=0x7fa1e802cba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 -- 172.21.15.96:0/2984584903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 msgr2=0x559a644c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 -- 172.21.15.96:0/2984584903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 msgr2=0x559a644bec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 -- 172.21.15.96:0/2984584903 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a64486500 con 0x559a644bf240 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f892d640 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c3d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40cc940 tx=0x7fa1e409b320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b817a49ad013afd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1da7fc640 1 -- 172.21.15.96:0/2984584903 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40b4070 con 0x559a644bf240 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1da7fc640 1 -- 172.21.15.96:0/2984584903 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e409b7e0 con 0x559a644bf240 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.315+0000 7fa1da7fc640 1 -- 172.21.15.96:0/2984584903 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4097e50 con 0x559a644bf240 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2984584903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 msgr2=0x559a644c3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c3d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40cc940 tx=0x7fa1e409b320 comp rx=0 tx=0).stop 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2984584903 shutdown_connections 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c3d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2984584903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644bec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2984584903 >> 172.21.15.96:0/2984584903 conn(0x559a644b9dc0 msgr2=0x559a644ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2984584903 shutdown_connections 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2984584903 wait complete. 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 -- start start 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.316+0000 7fa1fcf77740 1 Processor -- start 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 0x559a644c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559a644c7360 con 0x559a644c43b0 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a643df230 con 0x559a644be850 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a644c74e0 con 0x559a644bf240 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 10 client.0 _ll_register_callbacks cb 0x559a6441b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57568/0 (socket says 172.21.15.96:57568) 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 -- 172.21.15.96:0/2491213829 learned_addr learned my addr 172.21.15.96:0/2491213829 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 0x559a644c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 -- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 msgr2=0x559a644c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 -- 172.21.15.96:0/2491213829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 msgr2=0x559a644c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 0x559a644c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 -- 172.21.15.96:0/2491213829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a643ee100 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 0x559a644c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1f892d640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec011d10 tx=0x7fa1ec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9f1ad886be4fe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec01e060 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e409b7e0 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa1ec00fe00 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.317+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa1ec003970 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 10 client.17427 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559a641373f0 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e409b7e0 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fa1ec019070 con 0x559a644c43b0 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 10 client.17427 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 10 client.17427 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:47:42.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 20 client.17427 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a644d4cf0 0x559a644d7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 4 client.17427 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a644309d0 con 0x559a644d4cf0 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.318+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a644d4cf0 0x559a644d7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.319+0000 7fa1d37fe640 10 client.17427 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.319+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a644d4cf0 0x559a644d7130 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97768298ed187222 server_cookie=3da28eda00179b89 in_seq=0 out_seq=0 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.319+0000 7fa1d37fe640 1 -- 172.21.15.96:0/2491213829 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x559a644309d0 con 0x559a644d4cf0 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.319+0000 7fa1d37fe640 10 client.17427 handle_command_reply: tid=0 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.319+0000 7fa1fcf77740 1 client.17427 shutdown 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 msgr2=0x559a644d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec011d10 tx=0x7fa1ec00f790 comp rx=0 tx=0).stop 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 shutdown_connections 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a644d4cf0 0x559a644d7130 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644c43b0 0x559a644d02f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644bf240 0x559a644c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1d37fe640 0 client.17427 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/2491213829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644be850 0x559a644c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 >> 172.21.15.96:0/2491213829 conn(0x559a644b9dc0 msgr2=0x559a644b8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 shutdown_connections 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 1 -- 172.21.15.96:0/2491213829 wait complete. 2024-03-18T15:47:42.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.320+0000 7fa1fcf77740 20 client.17427 trim_cache size 0 max 16384 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 Processor -- start 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 -- start start 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644bf240 0x559a644d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644c43b0 0x559a644b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a6c40 con 0x559a644be850 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a8810 con 0x559a644bf240 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a7340 con 0x559a644c43b0 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57578/0 (socket says 172.21.15.96:57578) 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 -- 172.21.15.96:0/825998412 learned_addr learned my addr 172.21.15.96:0/825998412 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644bf240 0x559a644d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f892d640 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644c43b0 0x559a644b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 -- 172.21.15.96:0/825998412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644c43b0 msgr2=0x559a644b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644c43b0 0x559a644b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 -- 172.21.15.96:0/825998412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644bf240 msgr2=0x559a644d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644bf240 0x559a644d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 -- 172.21.15.96:0/825998412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a642017d0 con 0x559a644be850 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8000c00 tx=0x7fa1e8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203fd6a5fd73204 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1d9ffb640 1 -- 172.21.15.96:0/825998412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e805a020 con 0x559a644be850 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1d9ffb640 1 -- 172.21.15.96:0/825998412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8055310 con 0x559a644be850 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.334+0000 7fa1d9ffb640 1 -- 172.21.15.96:0/825998412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80055e0 con 0x559a644be850 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- 172.21.15.96:0/825998412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 msgr2=0x559a644b8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8000c00 tx=0x7fa1e8047000 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- 172.21.15.96:0/825998412 shutdown_connections 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644c43b0 0x559a644b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644bf240 0x559a644d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/825998412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644be850 0x559a644b8750 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- 172.21.15.96:0/825998412 >> 172.21.15.96:0/825998412 conn(0x559a644b9dc0 msgr2=0x559a644c8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- 172.21.15.96:0/825998412 shutdown_connections 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- 172.21.15.96:0/825998412 wait complete. 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.335+0000 7fa1fcf77740 1 -- start start 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 Processor -- start 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644be850 0x559a644cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a8140 con 0x559a644bf240 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a7a40 con 0x559a644c43b0 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f41a4c20 con 0x559a644be850 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 10 client.0 _ll_register_callbacks cb 0x7fa1ec009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57586/0 (socket says 172.21.15.96:57586) 2024-03-18T15:47:42.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 -- 172.21.15.96:0/1403104174 learned_addr learned my addr 172.21.15.96:0/1403104174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644be850 0x559a644cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f892d640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 -- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644be850 msgr2=0x559a644cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644be850 0x559a644cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 -- 172.21.15.96:0/1403104174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 msgr2=0x559a644cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 -- 172.21.15.96:0/1403104174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a644cf540 con 0x559a644bf240 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1f37fe640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4097c40 tx=0x7fa1e409b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af63ea0ed3aa166f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40b4040 con 0x559a644bf240 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e8055310 con 0x559a644bf240 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.336+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa1e40cce00 con 0x559a644bf240 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa1e40ae020 con 0x559a644bf240 2024-03-18T15:47:42.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 10 client.17433 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559a6453b100 con 0x559a644bf240 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e8055310 con 0x559a644bf240 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fa1e40a3af0 con 0x559a644bf240 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 10 client.17433 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 10 client.17433 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 20 client.17433 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a6453b6b0 0x559a6453dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 4 client.17433 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a6442ad20 con 0x559a6453b6b0 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.337+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a6453b6b0 0x559a6453dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1d37fe640 10 client.17433 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1f3fff640 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a6453b6b0 0x559a6453dad0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f10423ecc9acb52 server_cookie=db934e9559cea96f in_seq=0 out_seq=0 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1d37fe640 1 -- 172.21.15.96:0/1403104174 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a6442ad20 con 0x559a6453b6b0 2024-03-18T15:47:42.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1d37fe640 10 client.17433 handle_command_reply: tid=0 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 client.17433 shutdown 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 msgr2=0x559a644cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4097c40 tx=0x7fa1e409b450 comp rx=0 tx=0).stop 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 shutdown_connections 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x559a6453b6b0 0x559a6453dad0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a644c43b0 0x559a644cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559a644bf240 0x559a644cdf40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1d37fe640 0 client.17433 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 --2- 172.21.15.96:0/1403104174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a644be850 0x559a644cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 >> 172.21.15.96:0/1403104174 conn(0x559a644b9dc0 msgr2=0x559a644bd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 shutdown_connections 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 1 -- 172.21.15.96:0/1403104174 wait complete. 2024-03-18T15:47:42.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fcf77740 20 client.17433 trim_cache size 0 max 16384 2024-03-18T15:47:42.346 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:42.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.338+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 msgr2=0x7fa1dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 0x7fa1dc0a4cc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa1e80096c0 tx=0x7fa1e802cba0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 msgr2=0x7fa1f4196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400e9f0 tx=0x7fa1e400edf0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 shutdown_connections 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa1dc0a2800 0x7fa1dc0a4cc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1f4102290 0x7fa1f419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f41018d0 0x7fa1f4196f90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 --2- 172.21.15.96:0/308626030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1f41006e0 0x7fa1f4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 >> 172.21.15.96:0/308626030 conn(0x7fa1f40fbc70 msgr2=0x7fa1f4104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 shutdown_connections 2024-03-18T15:47:42.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.339+0000 7fa1fa390640 1 -- 172.21.15.96:0/308626030 wait complete. 2024-03-18T15:47:42.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:47:42.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.456+0000 7f67205f1640 1 -- 172.21.15.96:0/1029145156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67181006e0 msgr2=0x7f6718100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.456+0000 7f67205f1640 1 --2- 172.21.15.96:0/1029145156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67181006e0 0x7f6718100b00 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f671402bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:42.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- 172.21.15.96:0/1029145156 shutdown_connections 2024-03-18T15:47:42.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- 172.21.15.96:0/1029145156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6718102290 0x7f6718108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- 172.21.15.96:0/1029145156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 0x7f6718101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- 172.21.15.96:0/1029145156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67181006e0 0x7f6718100b00 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- 172.21.15.96:0/1029145156 >> 172.21.15.96:0/1029145156 conn(0x7f67180fbc70 msgr2=0x7f67180fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- 172.21.15.96:0/1029145156 shutdown_connections 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- 172.21.15.96:0/1029145156 wait complete. 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 Processor -- start 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- start start 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67181006e0 0x7f6718197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 0x7f67181980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f671810c330 con 0x7f6718102290 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f671810c1b0 con 0x7f67181018d0 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f67205f1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f671810c4b0 con 0x7f67181006e0 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f671eb8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f671eb8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57594/0 (socket says 172.21.15.96:57594) 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f671eb8e640 1 -- 172.21.15.96:0/1398453811 learned_addr learned my addr 172.21.15.96:0/1398453811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f671e38d640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67181006e0 0x7f6718197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.457+0000 7f671db8c640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 0x7f67181980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 -- 172.21.15.96:0/1398453811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67181006e0 msgr2=0x7f6718197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67181006e0 0x7f6718197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 -- 172.21.15.96:0/1398453811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 msgr2=0x7f67181980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 0x7f67181980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 -- 172.21.15.96:0/1398453811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671819f7d0 con 0x7f6718102290 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f671eb8e640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6710009fd0 tx=0x7f671000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f670f7fe640 1 -- 172.21.15.96:0/1398453811 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710017070 con 0x7f6718102290 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f670f7fe640 1 -- 172.21.15.96:0/1398453811 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6710003e50 con 0x7f6718102290 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671819fac0 con 0x7f6718102290 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f670f7fe640 1 -- 172.21.15.96:0/1398453811 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100106c0 con 0x7f6718102290 2024-03-18T15:47:42.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.458+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671819ffc0 con 0x7f6718102290 2024-03-18T15:47:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.460+0000 7f670f7fe640 1 -- 172.21.15.96:0/1398453811 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f6710003970 con 0x7f6718102290 2024-03-18T15:47:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 Processor -- start 2024-03-18T15:47:42.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 -- start start 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081723840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557081680490 con 0x55708171c240 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557081628c90 con 0x5570817213b0 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f67231d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55708167c530 con 0x55708171b850 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.461+0000 7f670f7fe640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 0x7f67040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57608/0 (socket says 172.21.15.96:57608) 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 -- 172.21.15.96:0/4191511032 learned_addr learned my addr 172.21.15.96:0/4191511032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671db8c640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671eb8e640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 0x7f67040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671db8c640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081723840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f670f7fe640 1 -- 172.21.15.96:0/1398453811 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f67100ce790 con 0x7f6718102290 2024-03-18T15:47:42.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 -- 172.21.15.96:0/4191511032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 msgr2=0x55708171bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 -- 172.21.15.96:0/4191511032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 msgr2=0x557081723840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081723840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 -- 172.21.15.96:0/4191511032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570816e3500 con 0x55708171c240 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671db8c640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671eb8e640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 0x7f67040a4cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67100cc5d0 tx=0x7f6710090690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f671e38d640 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f67140095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6151a6ecea3ed95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f670cbfe640 1 -- 172.21.15.96:0/4191511032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671402cb80 con 0x55708171c240 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f670cbfe640 1 -- 172.21.15.96:0/4191511032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714005040 con 0x55708171c240 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.462+0000 7f670cbfe640 1 -- 172.21.15.96:0/4191511032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714004920 con 0x55708171c240 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 -- 172.21.15.96:0/4191511032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 msgr2=0x557081720d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f67140095e0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 -- 172.21.15.96:0/4191511032 shutdown_connections 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081723840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x557081720d40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 --2- 172.21.15.96:0/4191511032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 -- 172.21.15.96:0/4191511032 >> 172.21.15.96:0/4191511032 conn(0x557081716dc0 msgr2=0x5570817171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 -- 172.21.15.96:0/4191511032 shutdown_connections 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.463+0000 7f67231d8740 1 -- 172.21.15.96:0/4191511032 wait complete. 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 -- start start 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 Processor -- start 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x557081725d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171c240 0x5570817262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557081724300 con 0x5570817213b0 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55708163c230 con 0x55708171b850 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557081724480 con 0x55708171c240 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f67231d8740 10 client.0 _ll_register_callbacks cb 0x557081678010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57622/0 (socket says 172.21.15.96:57622) 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 -- 172.21.15.96:0/235711796 learned_addr learned my addr 172.21.15.96:0/235711796 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671e38d640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171c240 0x5570817262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x557081725d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 -- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171c240 msgr2=0x5570817262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171c240 0x5570817262a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 -- 172.21.15.96:0/235711796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 msgr2=0x557081725d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x557081725d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 -- 172.21.15.96:0/235711796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708164b100 con 0x5570817213b0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f671db8c640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6708011c90 tx=0x7f67080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff233d6fc8a3ea38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.464+0000 7f6701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6708004250 con 0x5570817213b0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f67080043f0 con 0x5570817213b0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6714005040 con 0x5570817213b0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6708027030 con 0x5570817213b0 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 10 client.17445 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5570813943f0 con 0x5570817213b0 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714005040 con 0x5570817213b0 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f6708004830 con 0x5570817213b0 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 10 client.17445 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 10 client.17445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 20 client.17445 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x557081731c60 0x557081734120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 4 client.17445 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55708168d9d0 con 0x557081731c60 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.465+0000 7f671e38d640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x557081731c60 0x557081734120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.466+0000 7f671e38d640 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x557081731c60 0x557081734120 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a1c62a8d2b1bec2 server_cookie=f7021fdbfcee9919 in_seq=0 out_seq=0 2024-03-18T15:47:42.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.466+0000 7f6701ffb640 10 client.17445 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:42.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f6701ffb640 1 -- 172.21.15.96:0/235711796 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55708168d9d0 con 0x557081731c60 2024-03-18T15:47:42.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f6701ffb640 10 client.17445 handle_command_reply: tid=0 2024-03-18T15:47:42.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 client.17445 shutdown 2024-03-18T15:47:42.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 msgr2=0x55708172d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6708011c90 tx=0x7f67080046c0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 shutdown_connections 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x557081731c60 0x557081734120 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570817213b0 0x55708172d2a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171c240 0x5570817262a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f6701ffb640 0 client.17445 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 --2- 172.21.15.96:0/235711796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x557081725d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 >> 172.21.15.96:0/235711796 conn(0x557081716dc0 msgr2=0x557081715fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 shutdown_connections 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 1 -- 172.21.15.96:0/235711796 wait complete. 2024-03-18T15:47:42.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.468+0000 7f67231d8740 20 client.17445 trim_cache size 0 max 16384 2024-03-18T15:47:42.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 Processor -- start 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 -- start start 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081715750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67181a8d40 con 0x55708171c240 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f67181aa910 con 0x5570817213b0 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f67231d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67181a9440 con 0x55708171b850 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57638/0 (socket says 172.21.15.96:57638) 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.482+0000 7f671e38d640 1 -- 172.21.15.96:0/1731182400 learned_addr learned my addr 172.21.15.96:0/1731182400 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671db8c640 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081715750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671db8c640 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 -- 172.21.15.96:0/1731182400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 msgr2=0x55708171d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 -- 172.21.15.96:0/1731182400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 msgr2=0x557081715750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081715750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 -- 172.21.15.96:0/1731182400 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708145e7d0 con 0x55708171c240 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f671e38d640 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6714005ec0 tx=0x7f6714005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed26ec40f829532f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f6703fff640 1 -- 172.21.15.96:0/1731182400 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671404a070 con 0x55708171c240 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f6703fff640 1 -- 172.21.15.96:0/1731182400 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f671404fb60 con 0x55708171c240 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.483+0000 7f6703fff640 1 -- 172.21.15.96:0/1731182400 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671404f9c0 con 0x55708171c240 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- 172.21.15.96:0/1731182400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 msgr2=0x55708171dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f6714005ec0 tx=0x7f6714005e50 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- 172.21.15.96:0/1731182400 shutdown_connections 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5570817213b0 0x557081715750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x55708171dd10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- 172.21.15.96:0/1731182400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55708171b850 0x55708171d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- 172.21.15.96:0/1731182400 >> 172.21.15.96:0/1731182400 conn(0x557081716dc0 msgr2=0x5570817171a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- 172.21.15.96:0/1731182400 shutdown_connections 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- 172.21.15.96:0/1731182400 wait complete. 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- start start 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 Processor -- start 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x5570817931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570817213b0 0x55708179a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67181aa240 con 0x55708171c240 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f67181a9b40 con 0x55708171b850 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f67181a6d20 con 0x5570817213b0 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f67231d8740 10 client.0 _ll_register_callbacks cb 0x7f6708011ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f671e38d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57646/0 (socket says 172.21.15.96:57646) 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f671e38d640 1 -- 172.21.15.96:0/4241705921 learned_addr learned my addr 172.21.15.96:0/4241705921 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f671db8c640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570817213b0 0x55708179a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.484+0000 7f671db8c640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x5570817931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 -- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570817213b0 msgr2=0x55708179a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570817213b0 0x55708179a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 -- 172.21.15.96:0/4241705921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 msgr2=0x5570817931b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x5570817931b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 -- 172.21.15.96:0/4241705921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708179ad20 con 0x55708171c240 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671db8c640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x5570817931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671e38d640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f6714005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df30d6e9021acfc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671404a040 con 0x55708171c240 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f6714004230 con 0x55708171c240 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671404fb60 con 0x55708171c240 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f671404f390 con 0x55708171c240 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 10 client.17451 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55708179c020 con 0x55708171c240 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f671404fb60 con 0x55708171c240 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f6714034440 con 0x55708171c240 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 10 client.17451 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 10 client.17451 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 20 client.17451 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55708179c690 0x55708179ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 4 client.17451 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557081687d20 con 0x55708179c690 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.485+0000 7f671db8c640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55708179c690 0x55708179ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.486+0000 7f671db8c640 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55708179c690 0x55708179ea80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b5a90a68033c3cf server_cookie=a30f12a258e6fd46 in_seq=0 out_seq=0 2024-03-18T15:47:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.486+0000 7f6701ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:42.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f6701ffb640 1 -- 172.21.15.96:0/4241705921 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557081687d20 con 0x55708179c690 2024-03-18T15:47:42.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f6701ffb640 10 client.17451 handle_command_reply: tid=0 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 client.17451 shutdown 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 msgr2=0x5570817936f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6714009900 tx=0x7f6714005750 comp rx=0 tx=0).stop 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 shutdown_connections 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55708179c690 0x55708179ea80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5570817213b0 0x55708179a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55708171c240 0x5570817936f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f6701ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 --2- 172.21.15.96:0/4241705921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55708171b850 0x5570817931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 >> 172.21.15.96:0/4241705921 conn(0x557081716dc0 msgr2=0x557081725d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 shutdown_connections 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 1 -- 172.21.15.96:0/4241705921 wait complete. 2024-03-18T15:47:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.489+0000 7f67231d8740 20 client.17451 trim_cache size 0 max 16384 2024-03-18T15:47:42.495 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 msgr2=0x7f67040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 0x7f67040a4cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67100cc5d0 tx=0x7f6710090690 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 msgr2=0x7f671819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6710009fd0 tx=0x7f671000b260 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 shutdown_connections 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f67040a2800 0x7f67040a4cc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6718102290 0x7f671819f160 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67181018d0 0x7f67181980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 --2- 172.21.15.96:0/1398453811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67181006e0 0x7f6718197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 >> 172.21.15.96:0/1398453811 conn(0x7f67180fbc70 msgr2=0x7f67180689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 shutdown_connections 2024-03-18T15:47:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.490+0000 7f67205f1640 1 -- 172.21.15.96:0/1398453811 wait complete. 2024-03-18T15:47:42.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 -- 172.21.15.96:0/3114215998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95141018d0 msgr2=0x7f9514101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 --2- 172.21.15.96:0/3114215998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95141018d0 0x7f9514101d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9504009860 tx=0x7f950402bb70 comp rx=0 tx=0).stop 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 -- 172.21.15.96:0/3114215998 shutdown_connections 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 --2- 172.21.15.96:0/3114215998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9514102290 0x7f9514108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 --2- 172.21.15.96:0/3114215998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95141018d0 0x7f9514101d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 --2- 172.21.15.96:0/3114215998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 0x7f9514100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 -- 172.21.15.96:0/3114215998 >> 172.21.15.96:0/3114215998 conn(0x7f95140fbc70 msgr2=0x7f95140fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 -- 172.21.15.96:0/3114215998 shutdown_connections 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.608+0000 7f951b9a9640 1 -- 172.21.15.96:0/3114215998 wait complete. 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 Processor -- start 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 -- start start 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 0x7f9514199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9514102290 0x7f95141a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f951410b220 con 0x7f95141018d0 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f951410b0a0 con 0x7f9514102290 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f951b9a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f951410b3a0 con 0x7f95141006e0 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57666/0 (socket says 172.21.15.96:57666) 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 -- 172.21.15.96:0/2763598836 learned_addr learned my addr 172.21.15.96:0/2763598836 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9519f46640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9514102290 0x7f95141a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9519745640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 0x7f9514199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 -- 172.21.15.96:0/2763598836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 msgr2=0x7f9514199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 0x7f9514199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 -- 172.21.15.96:0/2763598836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9514102290 msgr2=0x7f95141a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9514102290 0x7f95141a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 -- 172.21.15.96:0/2763598836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95141a1980 con 0x7f95141018d0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.609+0000 7f9518f44640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9504005940 tx=0x7f9504032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.610+0000 7f950a7fc640 1 -- 172.21.15.96:0/2763598836 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504041020 con 0x7f95141018d0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.610+0000 7f950a7fc640 1 -- 172.21.15.96:0/2763598836 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950402c9f0 con 0x7f95141018d0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.610+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95141a1c10 con 0x7f95141018d0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.610+0000 7f950a7fc640 1 -- 172.21.15.96:0/2763598836 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9504039040 con 0x7f95141018d0 2024-03-18T15:47:42.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.610+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95141a2100 con 0x7f95141018d0 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f950a7fc640 1 -- 172.21.15.96:0/2763598836 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f9504044020 con 0x7f95141018d0 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 Processor -- start 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 -- start start 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 0x55d6b9ef7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef83b0 0x55d6b9efa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9e57490 con 0x55d6b9ef2850 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9dffc90 con 0x55d6b9ef3240 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f951e58c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9e53530 con 0x55d6b9ef83b0 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57672/0 (socket says 172.21.15.96:57672) 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f9519f46640 1 -- 172.21.15.96:0/2608865783 learned_addr learned my addr 172.21.15.96:0/2608865783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f9519745640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 0x55d6b9ef7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f9518f44640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef83b0 0x55d6b9efa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.613+0000 7f950a7fc640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 0x7f95000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 -- 172.21.15.96:0/2608865783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef83b0 msgr2=0x55d6b9efa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef83b0 0x55d6b9efa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 -- 172.21.15.96:0/2608865783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 msgr2=0x55d6b9ef2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 -- 172.21.15.96:0/2608865783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b9eba500 con 0x55d6b9ef3240 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519f46640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519f46640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 0x7f95000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f950a7fc640 1 -- 172.21.15.96:0/2763598836 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f95040f0410 con 0x7f95141018d0 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f9519745640 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 0x55d6b9ef7d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9510009bf0 tx=0x7f9510011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aebb1ac240d611a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f94ff7fe640 1 -- 172.21.15.96:0/2608865783 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510009da0 con 0x55d6b9ef3240 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f94ff7fe640 1 -- 172.21.15.96:0/2608865783 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9510005020 con 0x55d6b9ef3240 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.614+0000 7f94ff7fe640 1 -- 172.21.15.96:0/2608865783 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510004bc0 con 0x55d6b9ef3240 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- 172.21.15.96:0/2608865783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 msgr2=0x55d6b9ef7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 0x55d6b9ef7d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9510009bf0 tx=0x7f9510011810 comp rx=0 tx=0).stop 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f9519f46640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 0x7f95000a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55d6b9ef3aa0 tx=0x7f950c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- 172.21.15.96:0/2608865783 shutdown_connections 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef83b0 0x55d6b9efa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef3240 0x55d6b9ef7d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 --2- 172.21.15.96:0/2608865783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9ef2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- 172.21.15.96:0/2608865783 >> 172.21.15.96:0/2608865783 conn(0x55d6b9eeddc0 msgr2=0x55d6b9eee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- 172.21.15.96:0/2608865783 shutdown_connections 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- 172.21.15.96:0/2608865783 wait complete. 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 -- start start 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.615+0000 7f951e58c740 1 Processor -- start 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9efcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9efd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9efb360 con 0x55d6b9ef83b0 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9e13230 con 0x55d6b9ef2850 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d6b9efb4e0 con 0x55d6b9ef3240 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 10 client.0 _ll_register_callbacks cb 0x55d6b9e4f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519745640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9efcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57682/0 (socket says 172.21.15.96:57682) 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9518f44640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9efd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 -- 172.21.15.96:0/3284156248 learned_addr learned my addr 172.21.15.96:0/3284156248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 -- 172.21.15.96:0/3284156248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 msgr2=0x55d6b9efd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9efd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 -- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 msgr2=0x55d6b9efcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9efcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 -- 172.21.15.96:0/3284156248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b9e22100 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f9519f46640 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e6c0 tx=0x7f950c010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab4181080632927 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f94fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c029020 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f950c00dcb0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9510005020 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.616+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f950c034050 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 10 client.17463 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d6b9b6b3f0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9510005020 con 0x55d6b9ef83b0 2024-03-18T15:47:42.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f950c00b040 con 0x55d6b9ef83b0 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 10 client.17463 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 10 client.17463 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 20 client.17463 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f08b70 0x55d6b9f0b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 4 client.17463 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6b9e649d0 con 0x55d6b9f08b70 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f9519745640 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f08b70 0x55d6b9f0b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f9519745640 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f08b70 0x55d6b9f0b010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae684526c58970b7 server_cookie=3ec41dbbffe89006 in_seq=0 out_seq=0 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.617+0000 7f94fcff9640 10 client.17463 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.618+0000 7f94fcff9640 1 -- 172.21.15.96:0/3284156248 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d6b9e649d0 con 0x55d6b9f08b70 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.618+0000 7f94fcff9640 10 client.17463 handle_command_reply: tid=0 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.618+0000 7f951e58c740 1 client.17463 shutdown 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 msgr2=0x55d6b9f042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e6c0 tx=0x7f950c010040 comp rx=0 tx=0).stop 2024-03-18T15:47:42.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 shutdown_connections 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f042c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9efd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f08b70 0x55d6b9f0b010 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 --2- 172.21.15.96:0/3284156248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9efcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f94fcff9640 0 client.17463 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 >> 172.21.15.96:0/3284156248 conn(0x55d6b9eeddc0 msgr2=0x55d6b9eecfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 shutdown_connections 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 1 -- 172.21.15.96:0/3284156248 wait complete. 2024-03-18T15:47:42.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.619+0000 7f951e58c740 20 client.17463 trim_cache size 0 max 16384 2024-03-18T15:47:42.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 Processor -- start 2024-03-18T15:47:42.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 -- start start 2024-03-18T15:47:42.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f04280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f0fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ab010 con 0x55d6b9ef2850 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95141acbe0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f951e58c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ab710 con 0x55d6b9ef3240 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57698/0 (socket says 172.21.15.96:57698) 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 -- 172.21.15.96:0/1330654309 learned_addr learned my addr 172.21.15.96:0/1330654309 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9518f44640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f04280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519f46640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f0fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 -- 172.21.15.96:0/1330654309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 msgr2=0x55d6b9f04280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f04280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 -- 172.21.15.96:0/1330654309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 msgr2=0x55d6b9f0fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f0fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 -- 172.21.15.96:0/1330654309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6b9c357d0 con 0x55d6b9ef2850 2024-03-18T15:47:42.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519f46640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9518f44640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f04280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.633+0000 7f9519745640 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x55d6b9ef3cb0 tx=0x7f9510005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924dc14914649bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f94feffd640 1 -- 172.21.15.96:0/1330654309 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95100097a0 con 0x55d6b9ef2850 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f94feffd640 1 -- 172.21.15.96:0/1330654309 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95100052d0 con 0x55d6b9ef2850 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f94feffd640 1 -- 172.21.15.96:0/1330654309 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95100035f0 con 0x55d6b9ef2850 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 -- 172.21.15.96:0/1330654309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 msgr2=0x55d6b9f0f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x55d6b9ef3cb0 tx=0x7f9510005750 comp rx=0 tx=0).stop 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 -- 172.21.15.96:0/1330654309 shutdown_connections 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f0fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f04280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 --2- 172.21.15.96:0/1330654309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef2850 0x55d6b9f0f8d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 -- 172.21.15.96:0/1330654309 >> 172.21.15.96:0/1330654309 conn(0x55d6b9eeddc0 msgr2=0x55d6b9ef18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.634+0000 7f951e58c740 1 -- 172.21.15.96:0/1330654309 shutdown_connections 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 -- 172.21.15.96:0/1330654309 wait complete. 2024-03-18T15:47:42.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 -- start start 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 Processor -- start 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9f02050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f02590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95141ac510 con 0x55d6b9ef83b0 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95141abe10 con 0x55d6b9ef2850 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95141a8ff0 con 0x55d6b9ef3240 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f951e58c740 10 client.0 _ll_register_callbacks cb 0x7f950c00e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f9519f46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57704/0 (socket says 172.21.15.96:57704) 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f9519f46640 1 -- 172.21.15.96:0/1498487592 learned_addr learned my addr 172.21.15.96:0/1498487592 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f9518f44640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f02590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.635+0000 7f9519745640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9f02050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 -- 172.21.15.96:0/1498487592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 msgr2=0x55d6b9f02590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f02590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 -- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 msgr2=0x55d6b9f02050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9f02050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 -- 172.21.15.96:0/1498487592 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6b9f03b90 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f9519f46640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e690 tx=0x7f950c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d840ad51c4117 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c03d060 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f950c004ab0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95100052d0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f950c029040 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 10 client.17469 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d6b9f6f4d0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95100052d0 con 0x55d6b9ef83b0 2024-03-18T15:47:42.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7f950c005040 con 0x55d6b9ef83b0 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 10 client.17469 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 10 client.17469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 20 client.17469 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f6ff30 0x55d6b9f72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 4 client.17469 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.636+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6b9e5ed20 con 0x55d6b9f6ff30 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f9519745640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f6ff30 0x55d6b9f72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f94fcff9640 10 client.17469 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f9519745640 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f6ff30 0x55d6b9f72350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=92e00c2b1b00aca server_cookie=d776c5cf8f91c9fb in_seq=0 out_seq=0 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f94fcff9640 1 -- 172.21.15.96:0/1498487592 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6b9e5ed20 con 0x55d6b9f6ff30 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f94fcff9640 10 client.17469 handle_command_reply: tid=0 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 client.17469 shutdown 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 msgr2=0x55d6b9f035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e690 tx=0x7f950c005df0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 shutdown_connections 2024-03-18T15:47:42.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d6b9ef83b0 0x55d6b9f035a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d6b9ef3240 0x55d6b9f02590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55d6b9f6ff30 0x55d6b9f72350 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 --2- 172.21.15.96:0/1498487592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d6b9ef2850 0x55d6b9f02050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f94fcff9640 0 client.17469 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 >> 172.21.15.96:0/1498487592 conn(0x55d6b9eeddc0 msgr2=0x55d6b9eec750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 shutdown_connections 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 1 -- 172.21.15.96:0/1498487592 wait complete. 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.637+0000 7f951e58c740 20 client.17469 trim_cache size 0 max 16384 2024-03-18T15:47:42.645 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:42.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 msgr2=0x7f95000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 0x7f95000a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55d6b9ef3aa0 tx=0x7f950c004a40 comp rx=0 tx=0).stop 2024-03-18T15:47:42.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 msgr2=0x7f951419a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9504005940 tx=0x7f9504032470 comp rx=0 tx=0).stop 2024-03-18T15:47:42.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 shutdown_connections 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f95000a2800 0x7f95000a4cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9514102290 0x7f95141a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95141018d0 0x7f951419a2a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 --2- 172.21.15.96:0/2763598836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95141006e0 0x7f9514199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 >> 172.21.15.96:0/2763598836 conn(0x7f95140fbc70 msgr2=0x7f95140fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 shutdown_connections 2024-03-18T15:47:42.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.638+0000 7f951b9a9640 1 -- 172.21.15.96:0/2763598836 wait complete. 2024-03-18T15:47:42.650 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 -- 172.21.15.96:0/3199567625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c102290 msgr2=0x7fef0c108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 --2- 172.21.15.96:0/3199567625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c102290 0x7fef0c108b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ca0 tx=0x7fef0402c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 -- 172.21.15.96:0/3199567625 shutdown_connections 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 --2- 172.21.15.96:0/3199567625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c102290 0x7fef0c108b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 --2- 172.21.15.96:0/3199567625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1018d0 0x7fef0c101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 --2- 172.21.15.96:0/3199567625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c1006e0 0x7fef0c100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 -- 172.21.15.96:0/3199567625 >> 172.21.15.96:0/3199567625 conn(0x7fef0c0fbc70 msgr2=0x7fef0c0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 -- 172.21.15.96:0/3199567625 shutdown_connections 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 -- 172.21.15.96:0/3199567625 wait complete. 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.761+0000 7fef14268640 1 Processor -- start 2024-03-18T15:47:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 -- start start 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c1018d0 0x7fef0c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c102290 0x7fef0c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10c340 con 0x7fef0c1006e0 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10c1c0 con 0x7fef0c1018d0 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c10c4c0 con 0x7fef0c102290 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57716/0 (socket says 172.21.15.96:57716) 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 -- 172.21.15.96:0/1621983142 learned_addr learned my addr 172.21.15.96:0/1621983142 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef11803640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c1018d0 0x7fef0c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12805640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c102290 0x7fef0c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 -- 172.21.15.96:0/1621983142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c102290 msgr2=0x7fef0c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c102290 0x7fef0c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 -- 172.21.15.96:0/1621983142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c1018d0 msgr2=0x7fef0c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c1018d0 0x7fef0c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 -- 172.21.15.96:0/1621983142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c19f6b0 con 0x7fef0c1006e0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef12004640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d770 tx=0x7fef0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef037fe640 1 -- 172.21.15.96:0/1621983142 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08017070 con 0x7fef0c1006e0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c19f9a0 con 0x7fef0c1006e0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef037fe640 1 -- 172.21.15.96:0/1621983142 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef08009c30 con 0x7fef0c1006e0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.762+0000 7fef037fe640 1 -- 172.21.15.96:0/1621983142 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08003dd0 con 0x7fef0c1006e0 2024-03-18T15:47:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.763+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c19fef0 con 0x7fef0c1006e0 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.765+0000 7fef037fe640 1 -- 172.21.15.96:0/1621983142 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fef08003f70 con 0x7fef0c1006e0 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 Processor -- start 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 -- start start 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542588d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d54258b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d5424e8490 con 0x55d542583850 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d542490c90 con 0x55d5425893b0 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef16e4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d5424e4530 con 0x55d542584240 2024-03-18T15:47:42.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef037fe640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 0x7feef80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef12805640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef12004640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542588d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef12805640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57730/0 (socket says 172.21.15.96:57730) 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef11803640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d54258b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef12805640 1 -- 172.21.15.96:0/1236613714 learned_addr learned my addr 172.21.15.96:0/1236613714 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.766+0000 7fef12805640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 0x7feef80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef037fe640 1 -- 172.21.15.96:0/1621983142 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fef080ce360 con 0x7fef0c1006e0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 -- 172.21.15.96:0/1236613714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 msgr2=0x55d542588d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542588d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 -- 172.21.15.96:0/1236613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 msgr2=0x55d542583c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 -- 172.21.15.96:0/1236613714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d54254b500 con 0x55d5425893b0 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef12805640 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef11803640 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d54258b840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feefc009bf0 tx=0x7feefc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b26bb70fb1d29d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef12805640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 0x7feef80a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55d542584aa0 tx=0x7fef04032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef00bfe640 1 -- 172.21.15.96:0/1236613714 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004050 con 0x55d5425893b0 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef00bfe640 1 -- 172.21.15.96:0/1236613714 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feefc005500 con 0x55d5425893b0 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.767+0000 7fef00bfe640 1 -- 172.21.15.96:0/1236613714 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004e40 con 0x55d5425893b0 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- 172.21.15.96:0/1236613714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 msgr2=0x55d54258b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d54258b840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feefc009bf0 tx=0x7feefc011810 comp rx=0 tx=0).stop 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- 172.21.15.96:0/1236613714 shutdown_connections 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d54258b840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542588d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- 172.21.15.96:0/1236613714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542583c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- 172.21.15.96:0/1236613714 >> 172.21.15.96:0/1236613714 conn(0x55d54257edc0 msgr2=0x55d54257f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- 172.21.15.96:0/1236613714 shutdown_connections 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- 172.21.15.96:0/1236613714 wait complete. 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- start start 2024-03-18T15:47:42.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 Processor -- start 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d54258dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d542595290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.768+0000 7fef16e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d54258c360 con 0x55d542584240 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d5424a4230 con 0x55d542583850 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d54258c4e0 con 0x55d5425893b0 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 10 client.0 _ll_register_callbacks cb 0x55d5424e0010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef11803640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef11803640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57736/0 (socket says 172.21.15.96:57736) 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef11803640 1 -- 172.21.15.96:0/42391458 learned_addr learned my addr 172.21.15.96:0/42391458 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d54258dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12805640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d542595290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 -- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 msgr2=0x55d542595290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d542595290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 -- 172.21.15.96:0/42391458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 msgr2=0x55d54258e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 -- 172.21.15.96:0/42391458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5424b3100 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef11803640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef12004640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d54258dd80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d740 tx=0x7fef0809a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1d1a87850073f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7feef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08020040 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feefc005500 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fef080973f0 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fef080b9030 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 10 client.27284 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.769+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d5421fc3f0 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feefc005500 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7fef080cd030 con 0x55d542583850 2024-03-18T15:47:42.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 10 client.27284 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 10 client.27284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 20 client.27284 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d542599b20 0x55d54259bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 4 client.27284 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5424f59d0 con 0x55d542599b20 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef11803640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d542599b20 0x55d54259bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7feef5ffb640 10 client.27284 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.770+0000 7fef11803640 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d542599b20 0x55d54259bfe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190523135c6f7974 server_cookie=1d81dd1b7baf5c5e in_seq=0 out_seq=0 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.771+0000 7feef5ffb640 1 -- 172.21.15.96:0/42391458 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55d5424f59d0 con 0x55d542599b20 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.771+0000 7feef5ffb640 10 client.27284 handle_command_reply: tid=0 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 client.27284 shutdown 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 msgr2=0x55d54258dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d54258dd80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d740 tx=0x7fef0809a980 comp rx=0 tx=0).stop 2024-03-18T15:47:42.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 shutdown_connections 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d542595290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d542599b20 0x55d54259bfe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d54258e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 --2- 172.21.15.96:0/42391458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d54258dd80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7feef5ffb640 0 client.27284 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 >> 172.21.15.96:0/42391458 conn(0x55d54257edc0 msgr2=0x55d54257dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 shutdown_connections 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 1 -- 172.21.15.96:0/42391458 wait complete. 2024-03-18T15:47:42.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.772+0000 7fef16e4e740 20 client.27284 trim_cache size 0 max 16384 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 Processor -- start 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 -- start start 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542595250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d542585c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1a8da0 con 0x55d542583850 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1aa970 con 0x55d5425893b0 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef16e4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1a94a0 con 0x55d542584240 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef12004640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef12004640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57740/0 (socket says 172.21.15.96:57740) 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef12004640 1 -- 172.21.15.96:0/3274487154 learned_addr learned my addr 172.21.15.96:0/3274487154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef12805640 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d542585c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.785+0000 7fef11803640 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542595250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 -- 172.21.15.96:0/3274487154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 msgr2=0x55d542595250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542595250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 -- 172.21.15.96:0/3274487154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 msgr2=0x55d542585c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d542585c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 -- 172.21.15.96:0/3274487154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5422c67d0 con 0x55d542583850 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7fef12004640 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fef0809a450 tx=0x7fef08003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd58a22bb6d5a8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7feef7fff640 1 -- 172.21.15.96:0/3274487154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08020070 con 0x55d542583850 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7feef7fff640 1 -- 172.21.15.96:0/3274487154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0809ee20 con 0x55d542583850 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.786+0000 7feef7fff640 1 -- 172.21.15.96:0/3274487154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0809fd40 con 0x55d542583850 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- 172.21.15.96:0/3274487154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 msgr2=0x55d542585740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fef0809a450 tx=0x7fef08003870 comp rx=0 tx=0).stop 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- 172.21.15.96:0/3274487154 shutdown_connections 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d5425893b0 0x55d542585c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d542584240 0x55d542595250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- 172.21.15.96:0/3274487154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542583850 0x55d542585740 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- 172.21.15.96:0/3274487154 >> 172.21.15.96:0/3274487154 conn(0x55d54257edc0 msgr2=0x55d54259bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- 172.21.15.96:0/3274487154 shutdown_connections 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- 172.21.15.96:0/3274487154 wait complete. 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- start start 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 Processor -- start 2024-03-18T15:47:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d542595090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d5425920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1aa2a0 con 0x55d542584240 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1a9ba0 con 0x55d542583850 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c1a6d80 con 0x55d5425893b0 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef16e4e740 10 client.0 _ll_register_callbacks cb 0x7fef080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef11803640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef11803640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57756/0 (socket says 172.21.15.96:57756) 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef11803640 1 -- 172.21.15.96:0/2001416604 learned_addr learned my addr 172.21.15.96:0/2001416604 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef12004640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d542595090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.787+0000 7fef12805640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d5425920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 -- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 msgr2=0x55d5425920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d5425920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 -- 172.21.15.96:0/2001416604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 msgr2=0x55d542595090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d542595090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 -- 172.21.15.96:0/2001416604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5425fded0 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef12805640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d5425920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef11803640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feefc011d10 tx=0x7feefc005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec684d21b5b9493 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc02ecb0 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7feefc02e420 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef0809ee20 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feefc02ee50 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 10 client.17487 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d5425ff0d0 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef0809ee20 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7feefc039480 con 0x55d542584240 2024-03-18T15:47:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 10 client.17487 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 20 client.17487 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d5425ff970 0x55d542601d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 4 client.17487 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.788+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5424efd20 con 0x55d5425ff970 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef12004640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d5425ff970 0x55d542601d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7feef5ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef12004640 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d5425ff970 0x55d542601d90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89612e0470a5f343 server_cookie=28f2fa12ac346e26 in_seq=0 out_seq=0 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7feef5ffb640 1 -- 172.21.15.96:0/2001416604 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5424efd20 con 0x55d5425ff970 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7feef5ffb640 10 client.17487 handle_command_reply: tid=0 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 client.17487 shutdown 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 msgr2=0x55d542591b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7feefc011d10 tx=0x7feefc005a70 comp rx=0 tx=0).stop 2024-03-18T15:47:42.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 shutdown_connections 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d5425893b0 0x55d5425920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55d5425ff970 0x55d542601d90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d542584240 0x55d542591b80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 --2- 172.21.15.96:0/2001416604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d542583850 0x55d542595090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7feef5ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 >> 172.21.15.96:0/2001416604 conn(0x55d54257edc0 msgr2=0x55d542585740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 shutdown_connections 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 1 -- 172.21.15.96:0/2001416604 wait complete. 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.789+0000 7fef16e4e740 20 client.17487 trim_cache size 0 max 16384 2024-03-18T15:47:42.797 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 msgr2=0x7feef80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 0x7feef80a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x55d542584aa0 tx=0x7fef04032590 comp rx=0 tx=0).stop 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 msgr2=0x7fef0c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fef0800d770 tx=0x7fef0800dc30 comp rx=0 tx=0).stop 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 shutdown_connections 2024-03-18T15:47:42.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7feef80a2800 0x7feef80a4cc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef0c102290 0x7fef0c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef0c1018d0 0x7fef0c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 --2- 172.21.15.96:0/1621983142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c1006e0 0x7fef0c197a90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 >> 172.21.15.96:0/1621983142 conn(0x7fef0c0fbc70 msgr2=0x7fef0c068940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 shutdown_connections 2024-03-18T15:47:42.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:42.790+0000 7fef14268640 1 -- 172.21.15.96:0/1621983142 wait complete. 2024-03-18T15:47:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:42 smithi193 ceph-mon[36702]: pgmap v581: 97 pgs: 97 active+clean; 437 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 341 op/s 2024-03-18T15:47:42.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:42 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/896390512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:42 smithi096 ceph-mon[31930]: pgmap v581: 97 pgs: 97 active+clean; 437 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 341 op/s 2024-03-18T15:47:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:42 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/896390512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:43.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:42 smithi137 ceph-mon[35868]: pgmap v581: 97 pgs: 97 active+clean; 437 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 341 op/s 2024-03-18T15:47:43.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:42 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/896390512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:47:43.803 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:47:43.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:47:43.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:47:43.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.910+0000 7efd7aef8640 1 -- 172.21.15.96:0/3011267243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd74102290 msgr2=0x7efd74108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.910+0000 7efd7aef8640 1 --2- 172.21.15.96:0/3011267243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd74102290 0x7efd74108b20 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efd64009900 tx=0x7efd6402bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:43.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- 172.21.15.96:0/3011267243 shutdown_connections 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- 172.21.15.96:0/3011267243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd74102290 0x7efd74108b20 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- 172.21.15.96:0/3011267243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741018d0 0x7efd74101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- 172.21.15.96:0/3011267243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd741006e0 0x7efd74100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- 172.21.15.96:0/3011267243 >> 172.21.15.96:0/3011267243 conn(0x7efd740fbc70 msgr2=0x7efd740fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- 172.21.15.96:0/3011267243 shutdown_connections 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- 172.21.15.96:0/3011267243 wait complete. 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 Processor -- start 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- start start 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741006e0 0x7efd74197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd74102290 0x7efd7419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410d480 con 0x7efd741018d0 2024-03-18T15:47:43.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410d300 con 0x7efd741006e0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd7aef8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd7410d600 con 0x7efd74102290 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33440/0 (socket says 172.21.15.96:33440) 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd73fff640 1 -- 172.21.15.96:0/1683390751 learned_addr learned my addr 172.21.15.96:0/1683390751 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd78c94640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741006e0 0x7efd74197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.911+0000 7efd79495640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd74102290 0x7efd7419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 -- 172.21.15.96:0/1683390751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd74102290 msgr2=0x7efd7419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd74102290 0x7efd7419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 -- 172.21.15.96:0/1683390751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741006e0 msgr2=0x7efd74197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741006e0 0x7efd74197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 -- 172.21.15.96:0/1683390751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7419f6d0 con 0x7efd741018d0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd73fff640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9f0 tx=0x7efd6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd71ffb640 1 -- 172.21.15.96:0/1683390751 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6800cd80 con 0x7efd741018d0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7419f9c0 con 0x7efd741018d0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd71ffb640 1 -- 172.21.15.96:0/1683390751 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd68010910 con 0x7efd741018d0 2024-03-18T15:47:43.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd71ffb640 1 -- 172.21.15.96:0/1683390751 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd680188f0 con 0x7efd741018d0 2024-03-18T15:47:43.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.912+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7419ff10 con 0x7efd741018d0 2024-03-18T15:47:43.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.914+0000 7efd71ffb640 1 -- 172.21.15.96:0/1683390751 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efd68010ac0 con 0x7efd741018d0 2024-03-18T15:47:43.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 Processor -- start 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 -- start start 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24253db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd71ffb640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 0x7efd5c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d242594f0 0x557d2425b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d241b80d0 con 0x557d24254380 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557d24160c70 con 0x557d24253990 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d241b4170 con 0x557d242594f0 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd78c94640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 0x7efd5c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33446/0 (socket says 172.21.15.96:33446) 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 -- 172.21.15.96:0/3016534892 learned_addr learned my addr 172.21.15.96:0/3016534892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd79495640 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24253db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd79495640 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d242594f0 0x557d2425b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd71ffb640 1 -- 172.21.15.96:0/1683390751 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efd680cee40 con 0x7efd741018d0 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 -- 172.21.15.96:0/3016534892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d242594f0 msgr2=0x557d2425b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d242594f0 0x557d2425b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 -- 172.21.15.96:0/3016534892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 msgr2=0x557d24253db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24253db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 -- 172.21.15.96:0/3016534892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d23f947e0 con 0x557d24254380 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd78c94640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 0x7efd5c0a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd74101730 tx=0x7efd6c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.915+0000 7efd73fff640 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efd680cc910 tx=0x7efd6806b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11653d03d57d02a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd5affd640 1 -- 172.21.15.96:0/3016534892 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68026040 con 0x557d24254380 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd5affd640 1 -- 172.21.15.96:0/3016534892 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680a4e10 con 0x557d24254380 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd5affd640 1 -- 172.21.15.96:0/3016534892 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6809cd10 con 0x557d24254380 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3016534892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 msgr2=0x557d24258e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efd680cc910 tx=0x7efd6806b440 comp rx=0 tx=0).stop 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3016534892 shutdown_connections 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d242594f0 0x557d2425b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24254380 0x557d24258e80 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3016534892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24253db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3016534892 >> 172.21.15.96:0/3016534892 conn(0x557d2421bc50 msgr2=0x557d2424f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3016534892 shutdown_connections 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.916+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3016534892 wait complete. 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 -- start start 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 Processor -- start 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d2425e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d242594f0 0x557d242653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557d2425c440 con 0x557d24253990 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557d241740d0 con 0x557d242594f0 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557d2425c5c0 con 0x557d24254380 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd7dbbf740 10 client.0 _ll_register_callbacks cb 0x557d2413d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33452/0 (socket says 172.21.15.96:33452) 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 -- 172.21.15.96:0/2379018102 learned_addr learned my addr 172.21.15.96:0/2379018102 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd79495640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d242594f0 0x557d242653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd73fff640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d2425e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 -- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 msgr2=0x557d2425e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d2425e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 -- 172.21.15.96:0/2379018102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d242594f0 msgr2=0x557d242653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d242594f0 0x557d242653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.917+0000 7efd78c94640 1 -- 172.21.15.96:0/2379018102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d23ff4f70 con 0x557d24253990 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd78c94640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00eae0 tx=0x7efd6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95c8676ed919551 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01cac0 con 0x557d24253990 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efd6c01cc60 con 0x557d24253990 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd680a4e10 con 0x557d24253990 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efd6c02e030 con 0x557d24253990 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 10 client.17499 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:43.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557d23e8c170 con 0x557d24253990 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd680a4e10 con 0x557d24253990 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7efd6c0335c0 con 0x557d24253990 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 10 client.17499 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 20 client.17499 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d24269990 0x557d2426be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 4 client.17499 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.918+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d241c5610 con 0x557d24269990 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.919+0000 7efd73fff640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d24269990 0x557d2426be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.919+0000 7efd53fff640 10 client.17499 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.919+0000 7efd73fff640 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d24269990 0x557d2426be30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf79451d079a4ba server_cookie=a8dc92beaedca0f7 in_seq=0 out_seq=0 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd53fff640 1 -- 172.21.15.96:0/2379018102 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557d241c5610 con 0x557d24269990 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd53fff640 10 client.17499 handle_command_reply: tid=0 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 client.17499 shutdown 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 msgr2=0x557d2425dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00eae0 tx=0x7efd6c004ab0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 shutdown_connections 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d242594f0 0x557d242653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d2425e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d24269990 0x557d2426be30 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/2379018102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d24253990 0x557d2425dea0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd53fff640 0 client.17499 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 >> 172.21.15.96:0/2379018102 conn(0x557d2421bc50 msgr2=0x557d2424e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 shutdown_connections 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 1 -- 172.21.15.96:0/2379018102 wait complete. 2024-03-18T15:47:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.920+0000 7efd7dbbf740 20 client.17499 trim_cache size 0 max 16384 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 Processor -- start 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 -- start start 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24253990 0x557d2425e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24254380 0x557d24265370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd741a8e70 con 0x557d242594f0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd741aaa40 con 0x557d24254380 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd741a9570 con 0x557d24253990 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.937+0000 7efd79495640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33468/0 (socket says 172.21.15.96:33468) 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 -- 172.21.15.96:0/1349499861 learned_addr learned my addr 172.21.15.96:0/1349499861 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd78c94640 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24253990 0x557d2425e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd73fff640 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24254380 0x557d24265370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 -- 172.21.15.96:0/1349499861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24253990 msgr2=0x557d2425e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24253990 0x557d2425e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 -- 172.21.15.96:0/1349499861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24254380 msgr2=0x557d24265370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24254380 0x557d24265370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 -- 172.21.15.96:0/1349499861 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d24254760 con 0x557d242594f0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd79495640 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efd64009900 tx=0x7efd64005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadf2ae9977aa075 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd5a7fc640 1 -- 172.21.15.96:0/1349499861 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64004230 con 0x557d242594f0 2024-03-18T15:47:43.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd5a7fc640 1 -- 172.21.15.96:0/1349499861 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd640043d0 con 0x557d242594f0 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.938+0000 7efd5a7fc640 1 -- 172.21.15.96:0/1349499861 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd640054c0 con 0x557d242594f0 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- 172.21.15.96:0/1349499861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 msgr2=0x557d2424e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7efd64009900 tx=0x7efd64005b50 comp rx=0 tx=0).stop 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- 172.21.15.96:0/1349499861 shutdown_connections 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d2424e360 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24254380 0x557d24265370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/1349499861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24253990 0x557d2425e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- 172.21.15.96:0/1349499861 >> 172.21.15.96:0/1349499861 conn(0x557d2421bc50 msgr2=0x557d2424dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- 172.21.15.96:0/1349499861 shutdown_connections 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- 172.21.15.96:0/1349499861 wait complete. 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- start start 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 Processor -- start 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24264f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d242635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd741aa370 con 0x557d242594f0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd741a9c70 con 0x557d24253990 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd741a6e50 con 0x557d24254380 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.939+0000 7efd7dbbf740 10 client.0 _ll_register_callbacks cb 0x7efd6c00e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33476/0 (socket says 172.21.15.96:33476) 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 -- 172.21.15.96:0/3114255812 learned_addr learned my addr 172.21.15.96:0/3114255812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd73fff640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d242635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd78c94640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24264f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 -- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 msgr2=0x557d242635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d242635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 -- 172.21.15.96:0/3114255812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 msgr2=0x557d24264f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24264f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 -- 172.21.15.96:0/3114255812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d24264020 con 0x557d242594f0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd79495640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efd6402c7b0 tx=0x7efd6402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8280cff1a3dfefb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd53fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:43.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64052070 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efd64004ad0 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd640043d0 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efd64004cd0 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd7dbbf740 10 client.17505 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x557d242d09b0 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.940+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd640043d0 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7efd6404c420 con 0x557d242594f0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 10 client.17505 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 10 client.17505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 20 client.17505 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d242d13c0 0x557d242d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 4 client.17505 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557d241bf960 con 0x557d242d13c0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd78c94640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d242d13c0 0x557d242d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd53fff640 10 client.17505 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:43.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.941+0000 7efd78c94640 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d242d13c0 0x557d242d37e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819e36916daa4969 server_cookie=ae34524ca13faf7e in_seq=0 out_seq=0 2024-03-18T15:47:43.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd53fff640 1 -- 172.21.15.96:0/3114255812 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557d241bf960 con 0x557d242d13c0 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd53fff640 10 client.17505 handle_command_reply: tid=0 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 client.17505 shutdown 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 msgr2=0x557d24263ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7efd6402c7b0 tx=0x7efd6402c880 comp rx=0 tx=0).stop 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 shutdown_connections 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557d242d13c0 0x557d242d37e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557d242594f0 0x557d24263ae0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557d24254380 0x557d242635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 --2- 172.21.15.96:0/3114255812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557d24253990 0x557d24264f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 >> 172.21.15.96:0/3114255812 conn(0x557d2421bc50 msgr2=0x557d2424e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd53fff640 0 client.17505 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 shutdown_connections 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 1 -- 172.21.15.96:0/3114255812 wait complete. 2024-03-18T15:47:43.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.942+0000 7efd7dbbf740 20 client.17505 trim_cache size 0 max 16384 2024-03-18T15:47:43.949 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:47:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "1292ac5b-a47f-4922-ac84-e5df3815d00a", 2024-03-18T15:47:43.949 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:47:43.949 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 msgr2=0x7efd5c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 0x7efd5c0a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efd74101730 tx=0x7efd6c009500 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 msgr2=0x7efd74197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efd6800e9f0 tx=0x7efd6800eeb0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 shutdown_connections 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efd5c0a2800 0x7efd5c0a4cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd74102290 0x7efd7419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd741018d0 0x7efd74197ff0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 --2- 172.21.15.96:0/1683390751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd741006e0 0x7efd74197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 >> 172.21.15.96:0/1683390751 conn(0x7efd740fbc70 msgr2=0x7efd74104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 shutdown_connections 2024-03-18T15:47:43.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:43.943+0000 7efd7aef8640 1 -- 172.21.15.96:0/1683390751 wait complete. 2024-03-18T15:47:43.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:47:43.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:47:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 -- 172.21.15.96:0/1934688729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 msgr2=0x7ff1d8104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/1934688729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8104710 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009900 tx=0x7ff1d002bc30 comp rx=0 tx=0).stop 2024-03-18T15:47:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 -- 172.21.15.96:0/1934688729 shutdown_connections 2024-03-18T15:47:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/1934688729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8107580 0x7ff1d8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/1934688729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8104c50 0x7ff1d8107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/1934688729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8104710 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.061+0000 7ff1dd95b640 1 -- 172.21.15.96:0/1934688729 >> 172.21.15.96:0/1934688729 conn(0x7ff1d80fbc70 msgr2=0x7ff1d80fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- 172.21.15.96:0/1934688729 shutdown_connections 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- 172.21.15.96:0/1934688729 wait complete. 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 Processor -- start 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- start start 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8104c50 0x7ff1d81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8107580 0x7ff1d819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d810d460 con 0x7ff1d8102320 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d810d2e0 con 0x7ff1d8107580 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.062+0000 7ff1dd95b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d810d5e0 con 0x7ff1d8104c50 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33502/0 (socket says 172.21.15.96:33502) 2024-03-18T15:47:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/4057696972 learned_addr learned my addr 172.21.15.96:0/4057696972 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8107580 0x7ff1d819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8104c50 0x7ff1d81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/4057696972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8104c50 msgr2=0x7ff1d81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8104c50 0x7ff1d81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/4057696972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8107580 msgr2=0x7ff1d819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8107580 0x7ff1d819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/4057696972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d819f7b0 con 0x7ff1d8102320 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009230 tx=0x7ff1d002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1dc959640 1 -- 172.21.15.96:0/4057696972 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0041070 con 0x7ff1d8102320 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1dc959640 1 -- 172.21.15.96:0/4057696972 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1d0004450 con 0x7ff1d8102320 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d819fa40 con 0x7ff1d8102320 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1dc959640 1 -- 172.21.15.96:0/4057696972 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0003e20 con 0x7ff1d8102320 2024-03-18T15:47:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.063+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d819ff80 con 0x7ff1d8102320 2024-03-18T15:47:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1dc959640 1 -- 172.21.15.96:0/4057696972 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff1d0044070 con 0x7ff1d8102320 2024-03-18T15:47:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 Processor -- start 2024-03-18T15:47:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 -- start start 2024-03-18T15:47:44.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3170 0x556fe4ee3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 0x556fe4eeb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556fe4e48500 con 0x556fe4ee8cd0 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556fe4df0be0 con 0x556fe4ee3b60 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.066+0000 7ff1e0625740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556fe4e445a0 con 0x556fe4ee3170 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:37774/0 (socket says 172.21.15.96:37774) 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/1262909686 learned_addr learned my addr 172.21.15.96:0/1262909686 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3170 0x556fe4ee3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 0x556fe4eeb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1dc959640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 0x7ff1c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 -- 172.21.15.96:0/1262909686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3170 msgr2=0x556fe4ee3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 0x7ff1c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3170 0x556fe4ee3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 -- 172.21.15.96:0/1262909686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 msgr2=0x556fe4ee8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 -- 172.21.15.96:0/1262909686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe4e044b0 con 0x556fe4ee8cd0 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1dc959640 1 -- 172.21.15.96:0/4057696972 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff1d00f05a0 con 0x7ff1d8102320 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.067+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 0x556fe4eeb160 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80693e0 tx=0x7ff1cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4cc70ee410a127 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.068+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 0x7ff1c00a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x556fe4ee43c0 tx=0x7ff1c8004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.068+0000 7ff1bdffb640 1 -- 172.21.15.96:0/1262909686 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc009d70 con 0x556fe4ee8cd0 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.068+0000 7ff1bdffb640 1 -- 172.21.15.96:0/1262909686 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc005020 con 0x556fe4ee8cd0 2024-03-18T15:47:44.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.068+0000 7ff1bdffb640 1 -- 172.21.15.96:0/1262909686 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004b30 con 0x556fe4ee8cd0 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- 172.21.15.96:0/1262909686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 msgr2=0x556fe4eeb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 0x556fe4eeb160 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff1d80693e0 tx=0x7ff1cc011810 comp rx=0 tx=0).stop 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- 172.21.15.96:0/1262909686 shutdown_connections 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee8cd0 0x556fe4eeb160 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4ee8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1262909686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3170 0x556fe4ee3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- 172.21.15.96:0/1262909686 >> 172.21.15.96:0/1262909686 conn(0x556fe4ede6e0 msgr2=0x556fe4edeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- 172.21.15.96:0/1262909686 shutdown_connections 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- 172.21.15.96:0/1262909686 wait complete. 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 -- start start 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.069+0000 7ff1e0625740 1 Processor -- start 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 0x556fe4eedd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 0x556fe4ef5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556fe4eebc80 con 0x556fe4ef2d70 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556fe4e04160 con 0x556fe4ee3170 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556fe4eebe00 con 0x556fe4ee3b60 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 0x556fe4eedd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1e0625740 10 client.0 _ll_register_callbacks cb 0x556fe4de8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:37780/0 (socket says 172.21.15.96:37780) 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d67fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 0x556fe4eedd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39312/0 (socket says 172.21.15.96:39312) 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/3087336145 learned_addr learned my addr 172.21.15.96:0/3087336145 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 0x556fe4ef5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 -- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 msgr2=0x556fe4eedd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 0x556fe4eedd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 -- 172.21.15.96:0/3087336145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 msgr2=0x556fe4eed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d77fe640 1 -- 172.21.15.96:0/3087336145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe4c267d0 con 0x556fe4ef2d70 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.070+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 0x556fe4ef5160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8005fd0 tx=0x7ff1c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479e69a6c3dc0db0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8026030 con 0x556fe4ef2d70 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff1c801c030 con 0x556fe4ef2d70 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc005020 con 0x556fe4ef2d70 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff1c8013190 con 0x556fe4ef2d70 2024-03-18T15:47:44.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 10 client.17517 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556fe4b5c3f0 con 0x556fe4ef2d70 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc005020 con 0x556fe4ef2d70 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff1c8005190 con 0x556fe4ef2d70 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 10 client.17517 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 10 client.17517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 20 client.17517 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.071+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4efbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.072+0000 7ff1e0625740 4 client.17517 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.072+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe4e55a40 con 0x556fe4ef97a0 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.072+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4efbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.072+0000 7ff1b6ffd640 10 client.17517 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:44.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.072+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4efbc60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9803105582405d7 server_cookie=1cd494041d317607 in_seq=0 out_seq=0 2024-03-18T15:47:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.073+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/3087336145 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x556fe4e55a40 con 0x556fe4ef97a0 2024-03-18T15:47:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.073+0000 7ff1b6ffd640 10 client.17517 handle_command_reply: tid=0 2024-03-18T15:47:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.073+0000 7ff1e0625740 1 client.17517 shutdown 2024-03-18T15:47:44.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 msgr2=0x556fe4ef5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 0x556fe4ef5160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8005fd0 tx=0x7ff1c8004a40 comp rx=0 tx=0).stop 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 shutdown_connections 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4efbc60 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ef2d70 0x556fe4ef5160 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1b6ffd640 0 client.17517 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ee3b60 0x556fe4eedd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 --2- 172.21.15.96:0/3087336145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 >> 172.21.15.96:0/3087336145 conn(0x556fe4ede6e0 msgr2=0x556fe4edd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 shutdown_connections 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 1 -- 172.21.15.96:0/3087336145 wait complete. 2024-03-18T15:47:44.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.074+0000 7ff1e0625740 20 client.17517 trim_cache size 0 max 16384 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 Processor -- start 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 -- start start 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4edd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81a8e70 con 0x556fe4ee3b60 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81aaa40 con 0x556fe4ee3170 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1e0625740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81a9570 con 0x556fe4ef2d70 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33526/0 (socket says 172.21.15.96:33526) 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 -- 172.21.15.96:0/71865307 learned_addr learned my addr 172.21.15.96:0/71865307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4edd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 -- 172.21.15.96:0/71865307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 msgr2=0x556fe4edd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4edd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 -- 172.21.15.96:0/71865307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 msgr2=0x556fe4eec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 -- 172.21.15.96:0/71865307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe4b1c0f0 con 0x556fe4ee3b60 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.092+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc014040 tx=0x7ff1cc009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550cd053a4505079 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1bd7fa640 1 -- 172.21.15.96:0/71865307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc027070 con 0x556fe4ee3b60 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1bd7fa640 1 -- 172.21.15.96:0/71865307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc004510 con 0x556fe4ee3b60 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1bd7fa640 1 -- 172.21.15.96:0/71865307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004990 con 0x556fe4ee3b60 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 -- 172.21.15.96:0/71865307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 msgr2=0x556fe4eec710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc014040 tx=0x7ff1cc009930 comp rx=0 tx=0).stop 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 -- 172.21.15.96:0/71865307 shutdown_connections 2024-03-18T15:47:44.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4edd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3b60 0x556fe4eec710 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 --2- 172.21.15.96:0/71865307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3170 0x556fe4eec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 -- 172.21.15.96:0/71865307 >> 172.21.15.96:0/71865307 conn(0x556fe4ede6e0 msgr2=0x556fe4eed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 -- 172.21.15.96:0/71865307 shutdown_connections 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.093+0000 7ff1e0625740 1 -- 172.21.15.96:0/71865307 wait complete. 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 -- start start 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 Processor -- start 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4f0ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4f0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81aa370 con 0x556fe4ee3170 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81a9c70 con 0x556fe4ee3b60 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d81a6e50 con 0x556fe4ef2d70 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1e0625740 10 client.0 _ll_register_callbacks cb 0x7ff1c80139c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33538/0 (socket says 172.21.15.96:33538) 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/1968701435 learned_addr learned my addr 172.21.15.96:0/1968701435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4f0ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d77fe640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4f0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 msgr2=0x556fe4f0be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4f0be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/1968701435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 msgr2=0x556fe4f0ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4f0ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.094+0000 7ff1d6ffd640 1 -- 172.21.15.96:0/1968701435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe4e0f9d0 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1d6ffd640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00b9990 tx=0x7ff1d00f0990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3984b8018e4aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d004b040 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff1d00b9630 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1cc004510 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff1d00bcb00 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 10 client.17523 fetch_fsmap learned FSMap version 42 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556fe4f5f8c0 con 0x556fe4ee3170 2024-03-18T15:47:44.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1cc004510 con 0x556fe4ee3170 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 42) ==== 8374+0+0 (secure 0 0 0) 0x7ff1d00d2020 con 0x556fe4ee3170 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 10 client.17523 fetch_fsmap finished waiting for FSMap version 42 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 20 client.17523 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4f62330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 4 client.17523 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.095+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fe4e4fd90 con 0x556fe4ef97a0 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4f62330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1b6ffd640 10 client.17523 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1d67fc640 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4f62330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0bed5241e64b51 server_cookie=8900a2166d22b614 in_seq=0 out_seq=0 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1b6ffd640 1 -- 172.21.15.96:0/1968701435 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556fe4e4fd90 con 0x556fe4ef97a0 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1b6ffd640 10 client.17523 handle_command_reply: tid=0 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 client.17523 shutdown 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 msgr2=0x556fe4f0a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00b9990 tx=0x7ff1d00f0990 comp rx=0 tx=0).stop 2024-03-18T15:47:44.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 shutdown_connections 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556fe4ef2d70 0x556fe4f0be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556fe4ee3b60 0x556fe4f0ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x556fe4ef97a0 0x556fe4f62330 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 --2- 172.21.15.96:0/1968701435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe4ee3170 0x556fe4f0a930 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1b6ffd640 0 client.17523 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 >> 172.21.15.96:0/1968701435 conn(0x556fe4ede6e0 msgr2=0x556fe4eed770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 shutdown_connections 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.096+0000 7ff1e0625740 1 -- 172.21.15.96:0/1968701435 wait complete. 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1e0625740 20 client.17523 trim_cache size 0 max 16384 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:47:44.103 INFO:teuthology.orchestra.run.smithi096.stdout: "1292ac5b-a47f-4922-ac84-e5df3815d00a": { 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "1292ac5b-a47f-4922-ac84-e5df3815d00a", 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:47:44.104 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 msgr2=0x7ff1c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 0x7ff1c00a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x556fe4ee43c0 tx=0x7ff1c8004ab0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 msgr2=0x7ff1d8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009230 tx=0x7ff1d002c650 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 shutdown_connections 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff1c00a2880 0x7ff1c00a4d40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1d8107580 0x7ff1d819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d8104c50 0x7ff1d81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 --2- 172.21.15.96:0/4057696972 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1d8102320 0x7ff1d8197b90 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 >> 172.21.15.96:0/4057696972 conn(0x7ff1d80fbc70 msgr2=0x7ff1d8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 shutdown_connections 2024-03-18T15:47:44.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:47:44.097+0000 7ff1dd95b640 1 -- 172.21.15.96:0/4057696972 wait complete. 2024-03-18T15:47:44.108 INFO:tasks.cephfs.filesystem:scrub status for tag:1292ac5b-a47f-4922-ac84-e5df3815d00a - {'path': '/', 'tag': '1292ac5b-a47f-4922-ac84-e5df3815d00a', 'options': 'recursive,force'} 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: pgmap v582: 97 pgs: 97 active+clean; 432 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:44.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:44.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:44.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:44 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: pgmap v582: 97 pgs: 97 active+clean; 432 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:44.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:44.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:44.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:44 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: pgmap v582: 97 pgs: 97 active+clean; 432 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 402 op/s 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:45.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:44 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:45.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:45 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:47:45.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:45 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:47:46.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:45 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:47:46.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:46 smithi193 ceph-mon[36702]: pgmap v583: 97 pgs: 97 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-03-18T15:47:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:46 smithi096 ceph-mon[31930]: pgmap v583: 97 pgs: 97 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-03-18T15:47:47.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:46 smithi137 ceph-mon[35868]: pgmap v583: 97 pgs: 97 active+clean; 414 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-03-18T15:47:48.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:48 smithi193 ceph-mon[36702]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-03-18T15:47:48.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:48 smithi096 ceph-mon[31930]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-03-18T15:47:49.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:48 smithi137 ceph-mon[35868]: pgmap v584: 97 pgs: 97 active+clean; 412 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 390 op/s 2024-03-18T15:47:49.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:49 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:49.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:49 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:50.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:49 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:47:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:50 smithi193 ceph-mon[36702]: pgmap v585: 97 pgs: 97 active+clean; 390 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-03-18T15:47:50.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:50 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:50 smithi096 ceph-mon[31930]: pgmap v585: 97 pgs: 97 active+clean; 390 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-03-18T15:47:50.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:50 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:51.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:50 smithi137 ceph-mon[35868]: pgmap v585: 97 pgs: 97 active+clean; 390 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 752 op/s 2024-03-18T15:47:51.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:50 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:47:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:52 smithi096 ceph-mon[31930]: pgmap v586: 97 pgs: 97 active+clean; 382 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1011 KiB/s wr, 794 op/s 2024-03-18T15:47:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:52 smithi137 ceph-mon[35868]: pgmap v586: 97 pgs: 97 active+clean; 382 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1011 KiB/s wr, 794 op/s 2024-03-18T15:47:53.167 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:52 smithi193 ceph-mon[36702]: pgmap v586: 97 pgs: 97 active+clean; 382 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1011 KiB/s wr, 794 op/s 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.967+0000 7f2df83f1640 1 -- 172.21.15.96:0/1629363342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 msgr2=0x7f2df01017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.967+0000 7f2df83f1640 1 --2- 172.21.15.96:0/1629363342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01017f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009ca0 tx=0x7f2de802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- 172.21.15.96:0/1629363342 shutdown_connections 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- 172.21.15.96:0/1629363342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01017f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- 172.21.15.96:0/1629363342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df01009d0 0x7f2df0100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- 172.21.15.96:0/1629363342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0100070 0x7f2df0100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- 172.21.15.96:0/1629363342 >> 172.21.15.96:0/1629363342 conn(0x7f2df00fbc70 msgr2=0x7f2df00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- 172.21.15.96:0/1629363342 shutdown_connections 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- 172.21.15.96:0/1629363342 wait complete. 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 Processor -- start 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- start start 2024-03-18T15:47:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df0100070 0x7f2df0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df01009d0 0x7f2df019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f650 con 0x7f2df0101370 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f4d0 con 0x7f2df01009d0 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.968+0000 7f2df83f1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2df010f7d0 con 0x7f2df0100070 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39460/0 (socket says 172.21.15.96:39460) 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df618d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df0100070 0x7f2df0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 -- 172.21.15.96:0/2634231240 learned_addr learned my addr 172.21.15.96:0/2634231240 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df598c640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df01009d0 0x7f2df019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 -- 172.21.15.96:0/2634231240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df0100070 msgr2=0x7f2df0199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df0100070 0x7f2df0199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 -- 172.21.15.96:0/2634231240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df01009d0 msgr2=0x7f2df019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df01009d0 0x7f2df019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 -- 172.21.15.96:0/2634231240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a1800 con 0x7f2df0101370 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df698e640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2de8037040 tx=0x7f2de8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2de77fe640 1 -- 172.21.15.96:0/2634231240 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8041020 con 0x7f2df0101370 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2de77fe640 1 -- 172.21.15.96:0/2634231240 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8034cd0 con 0x7f2df0101370 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a1a90 con 0x7f2df0101370 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2de77fe640 1 -- 172.21.15.96:0/2634231240 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8039040 con 0x7f2df0101370 2024-03-18T15:47:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.969+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a1f80 con 0x7f2df0101370 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.971+0000 7f2de77fe640 1 -- 172.21.15.96:0/2634231240 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f2de8044020 con 0x7f2df0101370 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 Processor -- start 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2de77fe640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 0x7f2ddc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df598c640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 0x7f2ddc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2de77fe640 1 -- 172.21.15.96:0/2634231240 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2de80f0990 con 0x7f2df0101370 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 -- start start 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b92d30 0x55e821b97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b97ea0 0x55e821b9a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e821af7a40 con 0x55e821b92340 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e821aa0920 con 0x55e821b97ea0 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e821af3ae0 con 0x55e821b92d30 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39470/0 (socket says 172.21.15.96:39470) 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df598c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b92d30 0x55e821b97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df698e640 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b97ea0 0x55e821b9a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.972+0000 7f2df618d640 1 -- 172.21.15.96:0/2171441605 learned_addr learned my addr 172.21.15.96:0/2171441605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df598c640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 0x7f2ddc0a4d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2df019b0f0 tx=0x7f2de0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 -- 172.21.15.96:0/2171441605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b92d30 msgr2=0x55e821b97830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b92d30 0x55e821b97830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 -- 172.21.15.96:0/2171441605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b97ea0 msgr2=0x55e821b9a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b97ea0 0x55e821b9a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 -- 172.21.15.96:0/2171441605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e821ab8460 con 0x55e821b92340 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2df618d640 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009cf0 tx=0x7f2dec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721fc0073c7315fa server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2de4bfe640 1 -- 172.21.15.96:0/2171441605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec004030 con 0x55e821b92340 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2de4bfe640 1 -- 172.21.15.96:0/2171441605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec0041d0 con 0x55e821b92340 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.973+0000 7f2de4bfe640 1 -- 172.21.15.96:0/2171441605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0197a0 con 0x55e821b92340 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2171441605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 msgr2=0x55e821b92760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009cf0 tx=0x7f2dec011810 comp rx=0 tx=0).stop 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2171441605 shutdown_connections 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b97ea0 0x55e821b9a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b92d30 0x55e821b97830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2171441605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b92760 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2171441605 >> 172.21.15.96:0/2171441605 conn(0x55e821b8d8b0 msgr2=0x55e821b8dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2171441605 shutdown_connections 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2171441605 wait complete. 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 -- start start 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.974+0000 7f2dfafd8740 1 Processor -- start 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821b9cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e821b9ae50 con 0x55e821b92340 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e821ab3d70 con 0x55e821b92d30 2024-03-18T15:47:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e821b9afd0 con 0x55e821b97ea0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 10 client.0 _ll_register_callbacks cb 0x55e82180ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39484/0 (socket says 172.21.15.96:39484) 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 -- 172.21.15.96:0/2710267245 learned_addr learned my addr 172.21.15.96:0/2710267245 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df598c640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821b9cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df698e640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 -- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 msgr2=0x55e821ba3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 -- 172.21.15.96:0/2710267245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 msgr2=0x55e821b9cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821b9cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 -- 172.21.15.96:0/2710267245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e82180c210 con 0x55e821b92340 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2df618d640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00a010 tx=0x7f2dec005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638f9a5cc381a29b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec027070 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2dec005240 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec0041d0 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2dec033050 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 10 client.17535 fetch_fsmap learned FSMap version 43 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.975+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e8218d8620 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec0041d0 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f2dec019500 con 0x55e821b92340 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 10 client.17535 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 20 client.17535 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821ba8380 0x55e821baa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 4 client.17535 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:47:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e821b04f80 con 0x55e821ba8380 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2df598c640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821ba8380 0x55e821baa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2dd9ffb640 10 client.17535 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.976+0000 7f2df598c640 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821ba8380 0x55e821baa820 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622853a80278d37e server_cookie=568df184279ffd2f in_seq=0 out_seq=0 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.977+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/2710267245 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e821b04f80 con 0x55e821ba8380 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.977+0000 7f2dd9ffb640 10 client.17535 handle_command_reply: tid=0 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.977+0000 7f2dfafd8740 1 client.17535 shutdown 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 msgr2=0x55e821b9c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00a010 tx=0x7f2dec005d30 comp rx=0 tx=0).stop 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 shutdown_connections 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821b9cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821ba8380 0x55e821baa820 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/2710267245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b9c630 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:53.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dd9ffb640 0 client.17535 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 >> 172.21.15.96:0/2710267245 conn(0x55e821b8d8b0 msgr2=0x55e821b8cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 shutdown_connections 2024-03-18T15:47:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 1 -- 172.21.15.96:0/2710267245 wait complete. 2024-03-18T15:47:53.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.978+0000 7f2dfafd8740 20 client.17535 trim_cache size 0 max 16384 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.993+0000 7f2dfafd8740 1 Processor -- start 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.993+0000 7f2dfafd8740 1 -- start start 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821baf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01aaea0 con 0x55e821b92340 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01acb00 con 0x55e821b92d30 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01ab5a0 con 0x55e821b97ea0 2024-03-18T15:47:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df598c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39488/0 (socket says 172.21.15.96:39488) 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df598c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:33654/0 (socket says 172.21.15.96:33654) 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 -- 172.21.15.96:0/3395625883 learned_addr learned my addr 172.21.15.96:0/3395625883 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df698e640 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821baf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 -- 172.21.15.96:0/3395625883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 msgr2=0x55e821baf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821baf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 -- 172.21.15.96:0/3395625883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 msgr2=0x55e821ba3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 -- 172.21.15.96:0/3395625883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e821baaea0 con 0x55e821b92340 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2df618d640 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011850 tx=0x7f2dec003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97a23a35d8e6754 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2ddbfff640 1 -- 172.21.15.96:0/3395625883 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec027070 con 0x55e821b92340 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2ddbfff640 1 -- 172.21.15.96:0/3395625883 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec004000 con 0x55e821b92340 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.994+0000 7f2ddbfff640 1 -- 172.21.15.96:0/3395625883 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec019420 con 0x55e821b92340 2024-03-18T15:47:54.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3395625883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 msgr2=0x55e821b8ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011850 tx=0x7f2dec003d10 comp rx=0 tx=0).stop 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3395625883 shutdown_connections 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821baf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3395625883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821b8ca30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3395625883 >> 172.21.15.96:0/3395625883 conn(0x55e821b8d8b0 msgr2=0x55e821b8c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3395625883 shutdown_connections 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3395625883 wait complete. 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- start start 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 Processor -- start 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01ac430 con 0x55e821b92340 2024-03-18T15:47:54.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.995+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01abca0 con 0x55e821b92d30 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dfafd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01a8e80 con 0x55e821b97ea0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dfafd8740 10 client.0 _ll_register_callbacks cb 0x7f2de00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39490/0 (socket says 172.21.15.96:39490) 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 -- 172.21.15.96:0/3107176805 learned_addr learned my addr 172.21.15.96:0/3107176805 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df598c640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df698e640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 -- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 msgr2=0x55e821ba2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 -- 172.21.15.96:0/3107176805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 msgr2=0x55e821ba1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 -- 172.21.15.96:0/3107176805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e821b9af20 con 0x55e821b92340 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df598c640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2df618d640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2dec025040 tx=0x7f2dec005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5ab5e1e74b9917 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:47:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec027070 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f2dec019420 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec004000 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2dec047030 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dfafd8740 10 client.17541 fetch_fsmap learned FSMap version 43 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e821c0eac0 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.996+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec004000 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f2dec023b70 con 0x55e821b92340 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 10 client.17541 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 20 client.17541 populate_metadata read hostname 'smithi096' 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821c0f4c0 0x55e821c118e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 4 client.17541 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e821aff2d0 con 0x55e821c0f4c0 2024-03-18T15:47:54.003 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2df598c640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821c0f4c0 0x55e821c118e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:47:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2dd9ffb640 10 client.17541 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:54.004 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:53.997+0000 7f2df598c640 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821c0f4c0 0x55e821c118e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83da223cbcd1b2d server_cookie=e66f4dea110f0225 in_seq=0 out_seq=0 2024-03-18T15:47:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dd9ffb640 1 -- 172.21.15.96:0/3107176805 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e821aff2d0 con 0x55e821c0f4c0 2024-03-18T15:47:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dd9ffb640 10 client.17541 handle_command_reply: tid=0 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 client.17541 shutdown 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 msgr2=0x55e821ba18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2dec025040 tx=0x7f2dec005b90 comp rx=0 tx=0).stop 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 shutdown_connections 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e821b97ea0 0x55e821ba2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e821b92d30 0x55e821ba1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55e821c0f4c0 0x55e821c118e0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 --2- 172.21.15.96:0/3107176805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e821b92340 0x55e821ba18d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dd9ffb640 0 client.17541 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 >> 172.21.15.96:0/3107176805 conn(0x55e821b8d8b0 msgr2=0x55e821b8cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 shutdown_connections 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 1 -- 172.21.15.96:0/3107176805 wait complete. 2024-03-18T15:47:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.088+0000 7f2dfafd8740 20 client.17541 trim_cache size 0 max 16384 2024-03-18T15:47:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:47:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:47:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:47:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 msgr2=0x7f2ddc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 0x7f2ddc0a4d60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2df019b0f0 tx=0x7f2de0005f90 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 msgr2=0x7f2df01a1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2de8037040 tx=0x7f2de8032f40 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 shutdown_connections 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f2ddc0a28a0 0x7f2ddc0a4d60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2df0101370 0x7f2df01a1190 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df01009d0 0x7f2df019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 --2- 172.21.15.96:0/2634231240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2df0100070 0x7f2df0199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 >> 172.21.15.96:0/2634231240 conn(0x7f2df00fbc70 msgr2=0x7f2df00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 shutdown_connections 2024-03-18T15:47:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:47:54.089+0000 7f2df83f1640 1 -- 172.21.15.96:0/2634231240 wait complete. 2024-03-18T15:47:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:54 smithi096 ceph-mon[31930]: pgmap v587: 97 pgs: 97 active+clean; 376 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 802 op/s 2024-03-18T15:47:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:54 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:54.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:54 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:54 smithi137 ceph-mon[35868]: pgmap v587: 97 pgs: 97 active+clean; 376 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 802 op/s 2024-03-18T15:47:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:54 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:55.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:54 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:55.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:54 smithi193 ceph-mon[36702]: pgmap v587: 97 pgs: 97 active+clean; 376 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 802 op/s 2024-03-18T15:47:55.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:54 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:55.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:54 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:55 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:55.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:55 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:47:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:55 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:56.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:55 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:47:56.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:55 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:56.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:55 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:47:56.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:56 smithi096 ceph-mon[31930]: pgmap v588: 97 pgs: 97 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-03-18T15:47:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:56 smithi137 ceph-mon[35868]: pgmap v588: 97 pgs: 97 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-03-18T15:47:57.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:56 smithi193 ceph-mon[36702]: pgmap v588: 97 pgs: 97 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 815 op/s 2024-03-18T15:47:58.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:47:58 smithi096 ceph-mon[31930]: pgmap v589: 97 pgs: 97 active+clean; 366 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 905 KiB/s wr, 745 op/s 2024-03-18T15:47:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:47:58 smithi137 ceph-mon[35868]: pgmap v589: 97 pgs: 97 active+clean; 366 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 905 KiB/s wr, 745 op/s 2024-03-18T15:47:59.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:47:58 smithi193 ceph-mon[36702]: pgmap v589: 97 pgs: 97 active+clean; 366 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 905 KiB/s wr, 745 op/s 2024-03-18T15:48:00.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:00 smithi096 ceph-mon[31930]: pgmap v590: 97 pgs: 97 active+clean; 356 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 832 op/s 2024-03-18T15:48:00.969 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:00 smithi193 ceph-mon[36702]: pgmap v590: 97 pgs: 97 active+clean; 356 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 832 op/s 2024-03-18T15:48:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:00 smithi137 ceph-mon[35868]: pgmap v590: 97 pgs: 97 active+clean; 356 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 832 op/s 2024-03-18T15:48:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:02 smithi096 ceph-mon[31930]: pgmap v591: 97 pgs: 97 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1015 KiB/s wr, 500 op/s 2024-03-18T15:48:03.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:02 smithi137 ceph-mon[35868]: pgmap v591: 97 pgs: 97 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1015 KiB/s wr, 500 op/s 2024-03-18T15:48:03.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:02 smithi193 ceph-mon[36702]: pgmap v591: 97 pgs: 97 active+clean; 349 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1015 KiB/s wr, 500 op/s 2024-03-18T15:48:05.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:04 smithi137 ceph-mon[35868]: pgmap v592: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 409 op/s 2024-03-18T15:48:05.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:04 smithi193 ceph-mon[36702]: pgmap v592: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 409 op/s 2024-03-18T15:48:05.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:04 smithi096 ceph-mon[31930]: pgmap v592: 97 pgs: 97 active+clean; 346 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 409 op/s 2024-03-18T15:48:07.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:06 smithi137 ceph-mon[35868]: pgmap v593: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-03-18T15:48:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:06 smithi193 ceph-mon[36702]: pgmap v593: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-03-18T15:48:07.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:06 smithi096 ceph-mon[31930]: pgmap v593: 97 pgs: 97 active+clean; 329 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 415 op/s 2024-03-18T15:48:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 -- 172.21.15.96:0/2448943783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624100700 msgr2=0x7f5624100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 --2- 172.21.15.96:0/2448943783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624100700 0x7f5624100b20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5614009900 tx=0x7f561402bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 -- 172.21.15.96:0/2448943783 shutdown_connections 2024-03-18T15:48:07.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 --2- 172.21.15.96:0/2448943783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56241022b0 0x7f5624108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 --2- 172.21.15.96:0/2448943783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 0x7f5624101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 --2- 172.21.15.96:0/2448943783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624100700 0x7f5624100b20 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 -- 172.21.15.96:0/2448943783 >> 172.21.15.96:0/2448943783 conn(0x7f56240fbc70 msgr2=0x7f56240fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 -- 172.21.15.96:0/2448943783 shutdown_connections 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.200+0000 7f562a510640 1 -- 172.21.15.96:0/2448943783 wait complete. 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 Processor -- start 2024-03-18T15:48:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 -- start start 2024-03-18T15:48:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5624100700 0x7f56240699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 0x7f562419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f562410c370 con 0x7f56241022b0 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f562410c1f0 con 0x7f5624100700 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f562a510640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f562410c4f0 con 0x7f56241018f0 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54924/0 (socket says 172.21.15.96:54924) 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f5623fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5624100700 0x7f56240699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f5628aad640 1 -- 172.21.15.96:0/1937033008 learned_addr learned my addr 172.21.15.96:0/1937033008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.201+0000 7f56237fe640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 0x7f562419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 -- 172.21.15.96:0/1937033008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 msgr2=0x7f562419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 0x7f562419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 -- 172.21.15.96:0/1937033008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5624100700 msgr2=0x7f56240699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5624100700 0x7f56240699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 -- 172.21.15.96:0/1937033008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419f730 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f5628aad640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f561c00eb10 tx=0x7f561c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f56217fa640 1 -- 172.21.15.96:0/1937033008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004b20 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f56217fa640 1 -- 172.21.15.96:0/1937033008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c004cc0 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5624196630 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f56217fa640 1 -- 172.21.15.96:0/1937033008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c010760 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.202+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5624196b30 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f56217fa640 1 -- 172.21.15.96:0/1937033008 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f561c005020 con 0x7f56241022b0 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 Processor -- start 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 -- start start 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f56217fa640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 0x7f560c0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a48f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a50b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ed89a5a40 con 0x564ed8a49540 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564ed894e920 con 0x564ed8a48b50 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.205+0000 7f562d1da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ed89a1ae0 con 0x564ed8a4e6b0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56217fa640 1 -- 172.21.15.96:0/1937033008 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f561c0cf000 con 0x7f56241022b0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 0x7f560c0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54930/0 (socket says 172.21.15.96:54930) 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 -- 172.21.15.96:0/58590630 learned_addr learned my addr 172.21.15.96:0/58590630 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f5623fff640 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a48f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f5628aad640 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a50b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 -- 172.21.15.96:0/58590630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 msgr2=0x564ed8a50b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a50b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 -- 172.21.15.96:0/58590630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 msgr2=0x564ed8a48f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a48f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.206+0000 7f56237fe640 1 -- 172.21.15.96:0/58590630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed8966460 con 0x564ed8a49540 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.207+0000 7f56237fe640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 0x7f560c0a4d60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5618005d00 tx=0x7f561800bb90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.207+0000 7f56237fe640 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f56180123f0 tx=0x7f561801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f739ba6901176d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.207+0000 7f560a7fc640 1 -- 172.21.15.96:0/58590630 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618024070 con 0x564ed8a49540 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.207+0000 7f560a7fc640 1 -- 172.21.15.96:0/58590630 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5618010e00 con 0x564ed8a49540 2024-03-18T15:48:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.207+0000 7f560a7fc640 1 -- 172.21.15.96:0/58590630 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561800d750 con 0x564ed8a49540 2024-03-18T15:48:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- 172.21.15.96:0/58590630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 msgr2=0x564ed8a4e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f56180123f0 tx=0x7f561801c000 comp rx=0 tx=0).stop 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- 172.21.15.96:0/58590630 shutdown_connections 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a50b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a49540 0x564ed8a4e040 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 --2- 172.21.15.96:0/58590630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a48f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- 172.21.15.96:0/58590630 >> 172.21.15.96:0/58590630 conn(0x564ed8a3b8b0 msgr2=0x564ed8a3bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- 172.21.15.96:0/58590630 shutdown_connections 2024-03-18T15:48:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- 172.21.15.96:0/58590630 wait complete. 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.208+0000 7f562d1da740 1 -- start start 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 Processor -- start 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 0x564ed8a533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ed8a51600 con 0x564ed8a4e6b0 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564ed8961d70 con 0x564ed8a48b50 2024-03-18T15:48:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564ed8a51780 con 0x564ed8a49540 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54946/0 (socket says 172.21.15.96:54946) 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f5628aad640 1 -- 172.21.15.96:0/619406082 learned_addr learned my addr 172.21.15.96:0/619406082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f5623fff640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f562d1da740 10 client.0 _ll_register_callbacks cb 0x564ed86b9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.209+0000 7f56237fe640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 0x564ed8a533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 -- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 msgr2=0x564ed8a533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 0x564ed8a533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 -- 172.21.15.96:0/619406082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 msgr2=0x564ed8a52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a52eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 -- 172.21.15.96:0/619406082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed86ba210 con 0x564ed8a4e6b0 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f56237fe640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 0x564ed8a533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f5628aad640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ccad0 tx=0x7f561c09b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613bdc403d04167b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f56037fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a3e80 con 0x564ed8a4e6b0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f561c005310 con 0x564ed8a4e6b0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.210+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5618010e00 con 0x564ed8a4e6b0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f561c09b970 con 0x564ed8a4e6b0 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 10 client.17553 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ed8786620 con 0x564ed8a4e6b0 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5618010e00 con 0x564ed8a4e6b0 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f561c09adc0 con 0x564ed8a4e6b0 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 10 client.17553 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 20 client.17553 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8a61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 4 client.17553 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.211+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed89b2f80 con 0x564ed8a5ebf0 2024-03-18T15:48:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.212+0000 7f5623fff640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8a61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.212+0000 7f5623fff640 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8a61040 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5552375b3c193111 server_cookie=91089ec155bd48bf in_seq=0 out_seq=0 2024-03-18T15:48:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.212+0000 7f56037fe640 10 client.17553 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.215+0000 7f56037fe640 1 -- 172.21.15.96:0/619406082 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x564ed89b2f80 con 0x564ed8a5ebf0 2024-03-18T15:48:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.215+0000 7f56037fe640 10 client.17553 handle_command_reply: tid=0 2024-03-18T15:48:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.215+0000 7f562d1da740 1 client.17553 shutdown 2024-03-18T15:48:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.215+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 msgr2=0x564ed8a5a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.215+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ccad0 tx=0x7f561c09b570 comp rx=0 tx=0).stop 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 shutdown_connections 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8a61040 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a430 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f56037fe640 0 client.17553 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a49540 0x564ed8a533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 --2- 172.21.15.96:0/619406082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a48b50 0x564ed8a52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 >> 172.21.15.96:0/619406082 conn(0x564ed8a3b8b0 msgr2=0x564ed8a3abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 shutdown_connections 2024-03-18T15:48:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 1 -- 172.21.15.96:0/619406082 wait complete. 2024-03-18T15:48:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.216+0000 7f562d1da740 20 client.17553 trim_cache size 0 max 16384 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 Processor -- start 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 -- start start 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a48b50 0x564ed8a4a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a4ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a6b50 con 0x564ed8a4e6b0 2024-03-18T15:48:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a87b0 con 0x564ed8a49540 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f562d1da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a7250 con 0x564ed8a48b50 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f5628aad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54950/0 (socket says 172.21.15.96:54950) 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f5628aad640 1 -- 172.21.15.96:0/722207317 learned_addr learned my addr 172.21.15.96:0/722207317 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f5623fff640 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a48b50 0x564ed8a4a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.238+0000 7f56237fe640 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a4ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 -- 172.21.15.96:0/722207317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a48b50 msgr2=0x564ed8a4a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a48b50 0x564ed8a4a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 -- 172.21.15.96:0/722207317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 msgr2=0x564ed8a4ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a4ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 -- 172.21.15.96:0/722207317 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed8a37310 con 0x564ed8a4e6b0 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5628aad640 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f561c099c20 tx=0x7f561c099e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52e021b6a647c14 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5609ffb640 1 -- 172.21.15.96:0/722207317 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a4110 con 0x564ed8a4e6b0 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5609ffb640 1 -- 172.21.15.96:0/722207317 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c09adc0 con 0x564ed8a4e6b0 2024-03-18T15:48:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.239+0000 7f5609ffb640 1 -- 172.21.15.96:0/722207317 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c0860b0 con 0x564ed8a4e6b0 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 -- 172.21.15.96:0/722207317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 msgr2=0x564ed8a5a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f561c099c20 tx=0x7f561c099e80 comp rx=0 tx=0).stop 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 -- 172.21.15.96:0/722207317 shutdown_connections 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a4e6b0 0x564ed8a5a3f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a4ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 --2- 172.21.15.96:0/722207317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a48b50 0x564ed8a4a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 -- 172.21.15.96:0/722207317 >> 172.21.15.96:0/722207317 conn(0x564ed8a3b8b0 msgr2=0x564ed8a36a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 -- 172.21.15.96:0/722207317 shutdown_connections 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.240+0000 7f562d1da740 1 -- 172.21.15.96:0/722207317 wait complete. 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 -- start start 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 Processor -- start 2024-03-18T15:48:07.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 0x564ed8a57b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a59050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a80e0 con 0x564ed8a48b50 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a7950 con 0x564ed8a49540 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56241a4b30 con 0x564ed8a4e6b0 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.241+0000 7f562d1da740 10 client.0 _ll_register_callbacks cb 0x7f561c097c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:32868/0 (socket says 172.21.15.96:32868) 2024-03-18T15:48:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 -- 172.21.15.96:0/4026362170 learned_addr learned my addr 172.21.15.96:0/4026362170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f5623fff640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 0x564ed8a57b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f5628aad640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a59050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 -- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 msgr2=0x564ed8a59050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a59050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 -- 172.21.15.96:0/4026362170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 msgr2=0x564ed8a57b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 0x564ed8a57b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 -- 172.21.15.96:0/4026362170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed8a59640 con 0x564ed8a49540 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f5628aad640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a59050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f5623fff640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 0x564ed8a57b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56237fe640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x564ed8a49fb0 tx=0x7f561800b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2a108b40a24378 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.242+0000 7f56037fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5618024040 con 0x564ed8a49540 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561c09adc0 con 0x564ed8a49540 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f561801a040 con 0x564ed8a49540 2024-03-18T15:48:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5618005e50 con 0x564ed8a49540 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f562d1da740 10 client.27362 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564ed8ac5030 con 0x564ed8a49540 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.243+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c09adc0 con 0x564ed8a49540 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f5618039030 con 0x564ed8a49540 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 10 client.27362 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 10 client.27362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 20 client.27362 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8ac7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 4 client.27362 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ed89ad2d0 con 0x564ed8a5ebf0 2024-03-18T15:48:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.244+0000 7f5623fff640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8ac7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.246+0000 7f5623fff640 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8ac7980 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b514a132e67678ad server_cookie=4764863b93c08f87 in_seq=0 out_seq=0 2024-03-18T15:48:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.246+0000 7f56037fe640 10 client.27362 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f56037fe640 1 -- 172.21.15.96:0/4026362170 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ed89ad2d0 con 0x564ed8a5ebf0 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f56037fe640 10 client.27362 handle_command_reply: tid=0 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 client.27362 shutdown 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 msgr2=0x564ed8a58040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x564ed8a49fb0 tx=0x7f561800b3e0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 shutdown_connections 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564ed8a4e6b0 0x564ed8a59050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564ed8a49540 0x564ed8a58040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x564ed8a5ebf0 0x564ed8ac7980 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 --2- 172.21.15.96:0/4026362170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed8a48b50 0x564ed8a57b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f56037fe640 0 client.27362 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 >> 172.21.15.96:0/4026362170 conn(0x564ed8a3b8b0 msgr2=0x564ed8a52cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 shutdown_connections 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 1 -- 172.21.15.96:0/4026362170 wait complete. 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.259+0000 7f562d1da740 20 client.27362 trim_cache size 0 max 16384 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:48:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:48:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:48:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:48:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 msgr2=0x7f560c0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 0x7f560c0a4d60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5618005d00 tx=0x7f561800bb90 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 msgr2=0x7f562419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f561c00eb10 tx=0x7f561c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 shutdown_connections 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f560c0a28a0 0x7f560c0a4d60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56241022b0 0x7f562419ee20 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56241018f0 0x7f562419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 --2- 172.21.15.96:0/1937033008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5624100700 0x7f56240699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 >> 172.21.15.96:0/1937033008 conn(0x7f56240fbc70 msgr2=0x7f5624104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 shutdown_connections 2024-03-18T15:48:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:07.260+0000 7f562a510640 1 -- 172.21.15.96:0/1937033008 wait complete. 2024-03-18T15:48:09.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:08 smithi137 ceph-mon[35868]: pgmap v594: 97 pgs: 97 active+clean; 326 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 956 KiB/s wr, 340 op/s 2024-03-18T15:48:09.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:08 smithi193 ceph-mon[36702]: pgmap v594: 97 pgs: 97 active+clean; 326 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 956 KiB/s wr, 340 op/s 2024-03-18T15:48:09.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:08 smithi096 ceph-mon[31930]: pgmap v594: 97 pgs: 97 active+clean; 326 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 956 KiB/s wr, 340 op/s 2024-03-18T15:48:11.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:10 smithi137 ceph-mon[35868]: pgmap v595: 97 pgs: 97 active+clean; 293 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 420 op/s 2024-03-18T15:48:11.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:10 smithi193 ceph-mon[36702]: pgmap v595: 97 pgs: 97 active+clean; 293 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 420 op/s 2024-03-18T15:48:11.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:10 smithi096 ceph-mon[31930]: pgmap v595: 97 pgs: 97 active+clean; 293 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 420 op/s 2024-03-18T15:48:13.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:12 smithi137 ceph-mon[35868]: pgmap v596: 97 pgs: 97 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-03-18T15:48:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:12 smithi193 ceph-mon[36702]: pgmap v596: 97 pgs: 97 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-03-18T15:48:13.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:12 smithi096 ceph-mon[31930]: pgmap v596: 97 pgs: 97 active+clean; 289 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2024-03-18T15:48:14.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:48:14.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/2731419309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01006f0 msgr2=0x7fa5b0100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/2731419309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01006f0 0x7fa5b0100b10 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a402bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/2731419309 shutdown_connections 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/2731419309 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01022a0 0x7fa5b0108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/2731419309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/2731419309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01006f0 0x7fa5b0100b10 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/2731419309 >> 172.21.15.96:0/2731419309 conn(0x7fa5b00fbc80 msgr2=0x7fa5b00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/2731419309 shutdown_connections 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/2731419309 wait complete. 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 Processor -- start 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.217+0000 7fa5b6c1f640 1 -- start start 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01006f0 0x7fa5b0104680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b010c370 con 0x7fa5b01022a0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b010c1f0 con 0x7fa5b01018e0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b010c4f0 con 0x7fa5b01006f0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01006f0 0x7fa5b0104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55814/0 (socket says 172.21.15.96:55814) 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43542/0 (socket says 172.21.15.96:43542) 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3109997656 learned_addr learned my addr 172.21.15.96:0/3109997656 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3109997656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01006f0 msgr2=0x7fa5b0104680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01006f0 0x7fa5b0104680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3109997656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 msgr2=0x7fa5b0102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3109997656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b0103a90 con 0x7fa5b01022a0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800eaa0 tx=0x7fa5a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5adffb640 1 -- 172.21.15.96:0/3109997656 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800cdb0 con 0x7fa5b01022a0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5adffb640 1 -- 172.21.15.96:0/3109997656 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a8005020 con 0x7fa5b01022a0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b01a1c90 con 0x7fa5b01022a0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.218+0000 7fa5adffb640 1 -- 172.21.15.96:0/3109997656 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80116b0 con 0x7fa5b01022a0 2024-03-18T15:48:14.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.219+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b01a2190 con 0x7fa5b01022a0 2024-03-18T15:48:14.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.220+0000 7fa5adffb640 1 -- 172.21.15.96:0/3109997656 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fa5a8005560 con 0x7fa5b01022a0 2024-03-18T15:48:14.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 Processor -- start 2024-03-18T15:48:14.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 -- start start 2024-03-18T15:48:14.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5adffb640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 0x7fa5a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070b60 0x561f6c075660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c075cd0 0x561f6c078160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f6bfd5500 con 0x561f6c070b60 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561f6bf7dbe0 con 0x561f6c070170 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5b9805740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561f6bfd15a0 con 0x561f6c075cd0 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.221+0000 7fa5affff640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 0x7fa5a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5adffb640 1 -- 172.21.15.96:0/3109997656 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa5a80cec30 con 0x7fa5b01022a0 2024-03-18T15:48:14.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43554/0 (socket says 172.21.15.96:43554) 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 -- 172.21.15.96:0/1201109553 learned_addr learned my addr 172.21.15.96:0/1201109553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c075cd0 0x561f6c078160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5affff640 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070b60 0x561f6c075660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 -- 172.21.15.96:0/1201109553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c075cd0 msgr2=0x561f6c078160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c075cd0 0x561f6c078160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 -- 172.21.15.96:0/1201109553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070b60 msgr2=0x561f6c075660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070b60 0x561f6c075660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 -- 172.21.15.96:0/1201109553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6bf914b0 con 0x561f6c070170 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5affff640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 0x7fa5a00a4cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa5b01043f0 tx=0x7fa59c0128d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa6a76ffddaf3900 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa59affd640 1 -- 172.21.15.96:0/1201109553 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a402cb80 con 0x561f6c070170 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa59affd640 1 -- 172.21.15.96:0/1201109553 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4005040 con 0x561f6c070170 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.222+0000 7fa59affd640 1 -- 172.21.15.96:0/1201109553 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4004920 con 0x561f6c070170 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- 172.21.15.96:0/1201109553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 msgr2=0x561f6c070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009900 tx=0x7fa5a4005c00 comp rx=0 tx=0).stop 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- 172.21.15.96:0/1201109553 shutdown_connections 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c075cd0 0x561f6c078160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070b60 0x561f6c075660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 --2- 172.21.15.96:0/1201109553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c070590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- 172.21.15.96:0/1201109553 >> 172.21.15.96:0/1201109553 conn(0x561f6c06b6e0 msgr2=0x561f6c06bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- 172.21.15.96:0/1201109553 shutdown_connections 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- 172.21.15.96:0/1201109553 wait complete. 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 -- start start 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.223+0000 7fa5b9805740 1 Processor -- start 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 0x561f6c081bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f6c078c20 con 0x561f6c070170 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561f6bf91160 con 0x561f6c075cd0 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561f6c078da0 con 0x561f6c070b60 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 10 client.0 _ll_register_callbacks cb 0x561f6bf75150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55846/0 (socket says 172.21.15.96:55846) 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 -- 172.21.15.96:0/494015988 learned_addr learned my addr 172.21.15.96:0/494015988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 0x561f6c081bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5affff640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 -- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 msgr2=0x561f6c07abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 -- 172.21.15.96:0/494015988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 msgr2=0x561f6c081bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 0x561f6c081bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 -- 172.21.15.96:0/494015988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6bdb37d0 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 0x561f6c081bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa5a402c7b0 tx=0x7fa5a4005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277d5e569184344f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa58ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4004ad0 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa5a4004c70 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.224+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a4005040 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5a4004e70 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 10 client.17571 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561f6bce93f0 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4005040 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fa5a4045540 con 0x561f6c070170 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 10 client.17571 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 20 client.17571 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c085d50 0x561f6c088140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 4 client.17571 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f6bfe2a40 con 0x561f6c085d50 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.225+0000 7fa5affff640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c085d50 0x561f6c088140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa58ffff640 10 client.17571 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5affff640 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c085d50 0x561f6c088140 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af88e817cc7e070 server_cookie=4a4f72eda2c813f9 in_seq=0 out_seq=0 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa58ffff640 1 -- 172.21.15.96:0/494015988 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x561f6bfe2a40 con 0x561f6c085d50 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa58ffff640 10 client.17571 handle_command_reply: tid=0 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 client.17571 shutdown 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 msgr2=0x561f6c07a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fa5a402c7b0 tx=0x7fa5a4005510 comp rx=0 tx=0).stop 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 shutdown_connections 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c075cd0 0x561f6c081bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c085d50 0x561f6c088140 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 --2- 172.21.15.96:0/494015988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c070170 0x561f6c07a670 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa58ffff640 0 client.17571 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 >> 172.21.15.96:0/494015988 conn(0x561f6c06b6e0 msgr2=0x561f6c06a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.226+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 shutdown_connections 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.227+0000 7fa5b9805740 1 -- 172.21.15.96:0/494015988 wait complete. 2024-03-18T15:48:14.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.227+0000 7fa5b9805740 20 client.17571 trim_cache size 0 max 16384 2024-03-18T15:48:14.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 Processor -- start 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 -- start start 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c06bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c081bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01aafc0 con 0x561f6c075cd0 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01acc20 con 0x561f6c070170 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b9805740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01ab6c0 con 0x561f6c070b60 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c06bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5affff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c081bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55860/0 (socket says 172.21.15.96:55860) 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 -- 172.21.15.96:0/485727414 learned_addr learned my addr 172.21.15.96:0/485727414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b49bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c06bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43562/0 (socket says 172.21.15.96:43562) 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 -- 172.21.15.96:0/485727414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 msgr2=0x561f6c081bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c081bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 -- 172.21.15.96:0/485727414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 msgr2=0x561f6c06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c06bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 -- 172.21.15.96:0/485727414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6bca90f0 con 0x561f6c075cd0 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.244+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809b2f0 tx=0x7fa5a80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0129c8ecb81e4ab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa59a7fc640 1 -- 172.21.15.96:0/485727414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801f040 con 0x561f6c075cd0 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa59a7fc640 1 -- 172.21.15.96:0/485727414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a809b5f0 con 0x561f6c075cd0 2024-03-18T15:48:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa59a7fc640 1 -- 172.21.15.96:0/485727414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8084b80 con 0x561f6c075cd0 2024-03-18T15:48:14.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 -- 172.21.15.96:0/485727414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 msgr2=0x561f6c079c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809b2f0 tx=0x7fa5a80058d0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 -- 172.21.15.96:0/485727414 shutdown_connections 2024-03-18T15:48:14.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c079c90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c081bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 --2- 172.21.15.96:0/485727414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c06bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 -- 172.21.15.96:0/485727414 >> 172.21.15.96:0/485727414 conn(0x561f6c06b6e0 msgr2=0x561f6c06f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 -- 172.21.15.96:0/485727414 shutdown_connections 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.245+0000 7fa5b9805740 1 -- 172.21.15.96:0/485727414 wait complete. 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 -- start start 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 Processor -- start 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c07f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01ac550 con 0x561f6c075cd0 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01abdc0 con 0x561f6c070170 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b01a8fa0 con 0x561f6c070b60 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b9805740 10 client.0 _ll_register_callbacks cb 0x7fa5a4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55876/0 (socket says 172.21.15.96:55876) 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3384339283 learned_addr learned my addr 172.21.15.96:0/3384339283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c07f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5affff640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 msgr2=0x561f6c07f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3384339283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 msgr2=0x561f6c07f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c07f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.246+0000 7fa5b51bc640 1 -- 172.21.15.96:0/3384339283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6bf9c9d0 con 0x561f6c075cd0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b51bc640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80998a0 tx=0x7fa5a805aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74686dba9657e70f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80b3070 con 0x561f6c075cd0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fa5a8084d90 con 0x561f6c075cd0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a809b5f0 con 0x561f6c075cd0 2024-03-18T15:48:14.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5a801f030 con 0x561f6c075cd0 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 10 client.17577 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561f6c081ee0 con 0x561f6c075cd0 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a809b5f0 con 0x561f6c075cd0 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fa5a800f040 con 0x561f6c075cd0 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 10 client.17577 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 20 client.17577 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c0ec850 0x561f6c0eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 4 client.17577 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f6bfdcd90 con 0x561f6c0ec850 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.247+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c0ec850 0x561f6c0eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa58ffff640 10 client.17577 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:14.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b49bb640 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c0ec850 0x561f6c0eec70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c9deeaef6cbe0c6 server_cookie=fec0f25e5fd24f8 in_seq=0 out_seq=0 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa58ffff640 1 -- 172.21.15.96:0/3384339283 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f6bfdcd90 con 0x561f6c0ec850 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa58ffff640 10 client.17577 handle_command_reply: tid=0 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 client.17577 shutdown 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 msgr2=0x561f6c0807b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80998a0 tx=0x7fa5a805aef0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 shutdown_connections 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x561f6c0ec850 0x561f6c0eec70 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6c075cd0 0x561f6c0807b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa58ffff640 0 client.17577 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f6c070b60 0x561f6c07f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 --2- 172.21.15.96:0/3384339283 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561f6c070170 0x561f6c07f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 >> 172.21.15.96:0/3384339283 conn(0x561f6c06b6e0 msgr2=0x561f6c06f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 shutdown_connections 2024-03-18T15:48:14.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 1 -- 172.21.15.96:0/3384339283 wait complete. 2024-03-18T15:48:14.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.248+0000 7fa5b9805740 20 client.17577 trim_cache size 0 max 16384 2024-03-18T15:48:14.256 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:48:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:48:14.256 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:14.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 msgr2=0x7fa5a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 0x7fa5a00a4cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa5b01043f0 tx=0x7fa59c0128d0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 msgr2=0x7fa5b0103200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800eaa0 tx=0x7fa5a800ef60 comp rx=0 tx=0).stop 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 shutdown_connections 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fa5a00a2800 0x7fa5a00a4cc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5b01022a0 0x7fa5b0103200 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5b01018e0 0x7fa5b0102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 --2- 172.21.15.96:0/3109997656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b01006f0 0x7fa5b0104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 >> 172.21.15.96:0/3109997656 conn(0x7fa5b00fbc80 msgr2=0x7fa5b00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 shutdown_connections 2024-03-18T15:48:14.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.249+0000 7fa5b6c1f640 1 -- 172.21.15.96:0/3109997656 wait complete. 2024-03-18T15:48:14.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:48:14.259 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:48:14.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.365+0000 7fd7150a1640 1 -- 172.21.15.96:0/2816470206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd710101910 msgr2=0x7fd710101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.365+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2816470206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd710101910 0x7fd710101d90 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd704009900 tx=0x7fd70402bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 -- 172.21.15.96:0/2816470206 shutdown_connections 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2816470206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7101022d0 0x7fd710108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2816470206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd710101910 0x7fd710101d90 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2816470206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710100720 0x7fd710100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 -- 172.21.15.96:0/2816470206 >> 172.21.15.96:0/2816470206 conn(0x7fd7100fbc90 msgr2=0x7fd7100fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 -- 172.21.15.96:0/2816470206 shutdown_connections 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 -- 172.21.15.96:0/2816470206 wait complete. 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.366+0000 7fd7150a1640 1 Processor -- start 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 -- start start 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 0x7fd7101046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710101910 0x7fd710102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010c3a0 con 0x7fd7101022d0 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010c220 con 0x7fd710101910 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd7150a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010c520 con 0x7fd710100720 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710101910 0x7fd710102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70ed9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 0x7fd7101046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55900/0 (socket says 172.21.15.96:55900) 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 -- 172.21.15.96:0/2508485006 learned_addr learned my addr 172.21.15.96:0/2508485006 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 -- 172.21.15.96:0/2508485006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 msgr2=0x7fd7101046b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 0x7fd7101046b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 -- 172.21.15.96:0/2508485006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710101910 msgr2=0x7fd710102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710101910 0x7fd710102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 -- 172.21.15.96:0/2508485006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd710103770 con 0x7fd7101022d0 2024-03-18T15:48:14.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70ed9d640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 0x7fd7101046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd70f59e640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd70800eb10 tx=0x7fd70800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.367+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708004b20 con 0x7fd7101022d0 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.368+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd710103a60 con 0x7fd7101022d0 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.368+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd708004cc0 con 0x7fd7101022d0 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.368+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708010760 con 0x7fd7101022d0 2024-03-18T15:48:14.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.368+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a6600 con 0x7fd7101022d0 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.369+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7101a6890 con 0x7fd7101022d0 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.370+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd708005000 con 0x7fd7101022d0 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.371+0000 7fd6f7fff640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 0x7fd6f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.371+0000 7fd70ed9d640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 0x7fd6f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.371+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd7080cf4c0 con 0x7fd7101022d0 2024-03-18T15:48:14.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.371+0000 7fd70ed9d640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 0x7fd6f80a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd710101770 tx=0x7fd700005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.375+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fd70809c530 con 0x7fd7101022d0 2024-03-18T15:48:14.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.598+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7101a6a60 con 0x7fd7101022d0 2024-03-18T15:48:14.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.599+0000 7fd6f7fff640 1 -- 172.21.15.96:0/2508485006 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11946 (secure 0 0 0) 0x7fd70809be30 con 0x7fd7101022d0 2024-03-18T15:48:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:48:14.600 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:47:49.444355+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24472,24493,24485,24452,24460]},"id":1}]} 2024-03-18T15:48:14.601 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-03-18T15:48:14.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 msgr2=0x7fd6f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 0x7fd6f80a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd710101770 tx=0x7fd700005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 msgr2=0x7fd710103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd70800eb10 tx=0x7fd70800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 shutdown_connections 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd6f80a2800 0x7fd6f80a4cc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7101022d0 0x7fd710103230 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd710101910 0x7fd710102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 --2- 172.21.15.96:0/2508485006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd710100720 0x7fd7101046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 >> 172.21.15.96:0/2508485006 conn(0x7fd7100fbc90 msgr2=0x7fd7100fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 shutdown_connections 2024-03-18T15:48:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.601+0000 7fd7150a1640 1 -- 172.21.15.96:0/2508485006 wait complete. 2024-03-18T15:48:14.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.742+0000 7ff058197640 1 -- 172.21.15.96:0/3545562366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0501018d0 msgr2=0x7ff050101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.742+0000 7ff058197640 1 --2- 172.21.15.96:0/3545562366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0501018d0 0x7ff050101d50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- 172.21.15.96:0/3545562366 shutdown_connections 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- 172.21.15.96:0/3545562366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff050102290 0x7ff050108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- 172.21.15.96:0/3545562366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0501018d0 0x7ff050101d50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- 172.21.15.96:0/3545562366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501006e0 0x7ff050100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- 172.21.15.96:0/3545562366 >> 172.21.15.96:0/3545562366 conn(0x7ff0500fbc70 msgr2=0x7ff0500fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- 172.21.15.96:0/3545562366 shutdown_connections 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- 172.21.15.96:0/3545562366 wait complete. 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 Processor -- start 2024-03-18T15:48:14.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- start start 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 0x7ff050196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501018d0 0x7ff050196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff05010c340 con 0x7ff050102290 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff05010c1c0 con 0x7ff0501018d0 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.743+0000 7ff058197640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff05010c4c0 con 0x7ff0501006e0 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff055f33640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 0x7ff050196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff055732640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501018d0 0x7ff050196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff055f33640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 0x7ff050196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39288/0 (socket says 172.21.15.96:39288) 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55916/0 (socket says 172.21.15.96:55916) 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff055f33640 1 -- 172.21.15.96:0/2594316385 learned_addr learned my addr 172.21.15.96:0/2594316385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 -- 172.21.15.96:0/2594316385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 msgr2=0x7ff050196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 0x7ff050196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 -- 172.21.15.96:0/2594316385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501018d0 msgr2=0x7ff050196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501018d0 0x7ff050196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 -- 172.21.15.96:0/2594316385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019e630 con 0x7ff050102290 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff056734640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff048009fd0 tx=0x7ff04800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff046ffd640 1 -- 172.21.15.96:0/2594316385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04800be40 con 0x7ff050102290 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff046ffd640 1 -- 172.21.15.96:0/2594316385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff048017450 con 0x7ff050102290 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0500689a0 con 0x7ff050102290 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff046ffd640 1 -- 172.21.15.96:0/2594316385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048005280 con 0x7ff050102290 2024-03-18T15:48:14.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.744+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff050068ec0 con 0x7ff050102290 2024-03-18T15:48:14.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.746+0000 7ff046ffd640 1 -- 172.21.15.96:0/2594316385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7ff048004a40 con 0x7ff050102290 2024-03-18T15:48:14.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 Processor -- start 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 -- start start 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c74d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c753b0 0x55e432c77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e432bd4490 con 0x55e432c753b0 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e432b7cc90 con 0x55e432c6f850 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff05ad85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e432bd0530 con 0x55e432c70240 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff055732640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff055f33640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c74d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff055732640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43602/0 (socket says 172.21.15.96:43602) 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff055732640 1 -- 172.21.15.96:0/2667925968 learned_addr learned my addr 172.21.15.96:0/2667925968 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff046ffd640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 0x7ff03c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.747+0000 7ff055732640 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c753b0 0x55e432c77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff056734640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 0x7ff03c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 -- 172.21.15.96:0/2667925968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 msgr2=0x55e432c6fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 -- 172.21.15.96:0/2667925968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c753b0 msgr2=0x55e432c77840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c753b0 0x55e432c77840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 -- 172.21.15.96:0/2667925968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e432c37500 con 0x55e432c70240 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff046ffd640 1 -- 172.21.15.96:0/2594316385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff0480d0610 con 0x7ff050102290 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055732640 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff055f33640 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c74d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009cd0 tx=0x7ff04c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fcbb2a18d4c021a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff056734640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 0x7ff03c0a4d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0480999e0 tx=0x7ff04806fd40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff03bfff640 1 -- 172.21.15.96:0/2667925968 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c005020 con 0x55e432c70240 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff03bfff640 1 -- 172.21.15.96:0/2667925968 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c0047a0 con 0x55e432c70240 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.748+0000 7ff03bfff640 1 -- 172.21.15.96:0/2667925968 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c019650 con 0x55e432c70240 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- 172.21.15.96:0/2667925968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 msgr2=0x55e432c74d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c74d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009cd0 tx=0x7ff04c011810 comp rx=0 tx=0).stop 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- 172.21.15.96:0/2667925968 shutdown_connections 2024-03-18T15:48:14.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c753b0 0x55e432c77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c74d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2667925968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c6f850 0x55e432c6fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- 172.21.15.96:0/2667925968 >> 172.21.15.96:0/2667925968 conn(0x55e432c6adc0 msgr2=0x55e432c6b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- 172.21.15.96:0/2667925968 shutdown_connections 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- 172.21.15.96:0/2667925968 wait complete. 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 -- start start 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.749+0000 7ff05ad85740 1 Processor -- start 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e432c78360 con 0x55e432c70240 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e432b90230 con 0x55e432c753b0 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e432c784e0 con 0x55e432c6f850 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff05ad85740 10 client.0 _ll_register_callbacks cb 0x55e432bcc010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055f33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055f33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55938/0 (socket says 172.21.15.96:55938) 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055f33640 1 -- 172.21.15.96:0/2955624569 learned_addr learned my addr 172.21.15.96:0/2955624569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 -- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 msgr2=0x55e432c812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 -- 172.21.15.96:0/2955624569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 msgr2=0x55e432c7a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 -- 172.21.15.96:0/2955624569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e432b9f100 con 0x55e432c6f850 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055f33640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff055732640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79de0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55e432c70aa0 tx=0x7ff04002bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8d8a5034b144ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.750+0000 7ff0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040003c30 con 0x55e432c6f850 2024-03-18T15:48:14.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04c0047a0 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff040003dd0 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff04002cce0 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 10 client.27427 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e4328e83f0 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c0047a0 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7ff040056070 con 0x55e432c6f850 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 10 client.27427 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 10 client.27427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 20 client.27427 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432c85ae0 0x55e432c87f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 4 client.27427 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.751+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e432be19d0 con 0x55e432c85ae0 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.752+0000 7ff055f33640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432c85ae0 0x55e432c87f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.752+0000 7ff0397fa640 10 client.27427 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:14.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.752+0000 7ff055f33640 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432c85ae0 0x55e432c87f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8595c3d1c08123a0 server_cookie=45ef4b0b2a93a083 in_seq=0 out_seq=0 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff0397fa640 1 -- 172.21.15.96:0/2955624569 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55e432be19d0 con 0x55e432c85ae0 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff0397fa640 10 client.27427 handle_command_reply: tid=0 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 client.27427 shutdown 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 msgr2=0x55e432c79de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79de0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55e432c70aa0 tx=0x7ff04002bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 shutdown_connections 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432c85ae0 0x55e432c87f80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c7a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2955624569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79de0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff0397fa640 0 client.27427 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 >> 172.21.15.96:0/2955624569 conn(0x55e432c6adc0 msgr2=0x55e432c69fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 shutdown_connections 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 1 -- 172.21.15.96:0/2955624569 wait complete. 2024-03-18T15:48:14.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.753+0000 7ff05ad85740 20 client.27427 trim_cache size 0 max 16384 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 Processor -- start 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 -- start start 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a6a40 con 0x55e432c70240 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a86a0 con 0x55e432c753b0 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff05ad85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a7140 con 0x55e432c6f850 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff055f33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff055f33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55940/0 (socket says 172.21.15.96:55940) 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.766+0000 7ff055f33640 1 -- 172.21.15.96:0/3178244136 learned_addr learned my addr 172.21.15.96:0/3178244136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055732640 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 -- 172.21.15.96:0/3178244136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 msgr2=0x55e432c79d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 -- 172.21.15.96:0/3178244136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 msgr2=0x55e432c7a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c7a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 -- 172.21.15.96:0/3178244136 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4329b27d0 con 0x55e432c70240 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff055f33640 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01b040 tx=0x7ff04c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b372426f59c4db server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff03b7fe640 1 -- 172.21.15.96:0/3178244136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c004680 con 0x55e432c70240 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff03b7fe640 1 -- 172.21.15.96:0/3178244136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c004020 con 0x55e432c70240 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.767+0000 7ff03b7fe640 1 -- 172.21.15.96:0/3178244136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c02c600 con 0x55e432c70240 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- 172.21.15.96:0/3178244136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 msgr2=0x55e432c81270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01b040 tx=0x7ff04c005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- 172.21.15.96:0/3178244136 shutdown_connections 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432c7a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c70240 0x55e432c81270 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- 172.21.15.96:0/3178244136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c6f850 0x55e432c79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- 172.21.15.96:0/3178244136 >> 172.21.15.96:0/3178244136 conn(0x55e432c6adc0 msgr2=0x55e432c71780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- 172.21.15.96:0/3178244136 shutdown_connections 2024-03-18T15:48:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- 172.21.15.96:0/3178244136 wait complete. 2024-03-18T15:48:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- start start 2024-03-18T15:48:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 Processor -- start 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432ce9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a7fd0 con 0x55e432c6f850 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a7840 con 0x55e432c753b0 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0501a4a20 con 0x55e432c70240 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff05ad85740 10 client.0 _ll_register_callbacks cb 0x7ff0400044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.768+0000 7ff055732640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55948/0 (socket says 172.21.15.96:55948) 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 -- 172.21.15.96:0/2282077556 learned_addr learned my addr 172.21.15.96:0/2282077556 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432ce9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 -- 172.21.15.96:0/2282077556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 msgr2=0x55e432c7e420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c7e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 -- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 msgr2=0x55e432ce9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432ce9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 -- 172.21.15.96:0/2282077556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e432c814f0 con 0x55e432c6f850 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff055732640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff040005bd0 tx=0x7ff040005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d432682d237edc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040058690 con 0x55e432c6f850 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7ff040058830 con 0x55e432c6f850 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04c004020 con 0x55e432c6f850 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff040003e50 con 0x55e432c6f850 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff05ad85740 10 client.17601 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e432ceb3f0 con 0x55e432c6f850 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.769+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c004020 con 0x55e432c6f850 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7ff040056070 con 0x55e432c6f850 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 10 client.17601 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 10 client.17601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 20 client.17601 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432cebd40 0x55e432cee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 4 client.17601 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.770+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e432bdbd20 con 0x55e432cebd40 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.771+0000 7ff055f33640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432cebd40 0x55e432cee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.772+0000 7ff0397fa640 10 client.17601 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.772+0000 7ff055f33640 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432cebd40 0x55e432cee160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2aeb96a57630ebaa server_cookie=89662cd8007dd6eb in_seq=0 out_seq=0 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.772+0000 7ff0397fa640 1 -- 172.21.15.96:0/2282077556 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e432bdbd20 con 0x55e432cebd40 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.772+0000 7ff0397fa640 10 client.17601 handle_command_reply: tid=0 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.772+0000 7ff05ad85740 1 client.17601 shutdown 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 msgr2=0x55e432c7dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff040005bd0 tx=0x7ff040005fd0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 shutdown_connections 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55e432cebd40 0x55e432cee160 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e432c753b0 0x55e432ce9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e432c70240 0x55e432c7e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 --2- 172.21.15.96:0/2282077556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e432c6f850 0x55e432c7dee0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff0397fa640 0 client.17601 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 >> 172.21.15.96:0/2282077556 conn(0x55e432c6adc0 msgr2=0x55e432c79d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 shutdown_connections 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 1 -- 172.21.15.96:0/2282077556 wait complete. 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff05ad85740 20 client.17601 trim_cache size 0 max 16384 2024-03-18T15:48:14.777 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 msgr2=0x7ff03c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 0x7ff03c0a4d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff0480999e0 tx=0x7ff04806fd40 comp rx=0 tx=0).stop 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 msgr2=0x7ff05019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff048009fd0 tx=0x7ff04800b690 comp rx=0 tx=0).stop 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 shutdown_connections 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7ff03c0a2880 0x7ff03c0a4d40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff050102290 0x7ff05019dfc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0501018d0 0x7ff050196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 --2- 172.21.15.96:0/2594316385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0501006e0 0x7ff050196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.773+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 >> 172.21.15.96:0/2594316385 conn(0x7ff0500fbc70 msgr2=0x7ff050104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.774+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 shutdown_connections 2024-03-18T15:48:14.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.774+0000 7ff058197640 1 -- 172.21.15.96:0/2594316385 wait complete. 2024-03-18T15:48:14.785 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:48:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- 172.21.15.96:0/1260908817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01033a0 msgr2=0x7f71e0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 --2- 172.21.15.96:0/1260908817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01033a0 0x7f71e0109c30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009ca0 tx=0x7f71d802c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- 172.21.15.96:0/1260908817 shutdown_connections 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 --2- 172.21.15.96:0/1260908817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01033a0 0x7f71e0109c30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 --2- 172.21.15.96:0/1260908817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 0x7f71e0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 --2- 172.21.15.96:0/1260908817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01017f0 0x7f71e0101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- 172.21.15.96:0/1260908817 >> 172.21.15.96:0/1260908817 conn(0x7f71e00fcd60 msgr2=0x7f71e00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- 172.21.15.96:0/1260908817 shutdown_connections 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- 172.21.15.96:0/1260908817 wait complete. 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 Processor -- start 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.893+0000 7f71e86ac640 1 -- start start 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 0x7f71e019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01033a0 0x7f71e01a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010d470 con 0x7f71e01017f0 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010d2f0 con 0x7f71e01033a0 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010d5f0 con 0x7f71e01029e0 2024-03-18T15:48:14.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55970/0 (socket says 172.21.15.96:55970) 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 -- 172.21.15.96:0/3470730103 learned_addr learned my addr 172.21.15.96:0/3470730103 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e5c47640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 0x7f71e019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6c49640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01033a0 0x7f71e01a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 -- 172.21.15.96:0/3470730103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 msgr2=0x7f71e019a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 0x7f71e019a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 -- 172.21.15.96:0/3470730103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01033a0 msgr2=0x7f71e01a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01033a0 0x7f71e01a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 -- 172.21.15.96:0/3470730103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01a1920 con 0x7f71e01017f0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e6448640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e9c0 tx=0x7f71dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71d77fe640 1 -- 172.21.15.96:0/3470730103 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00cde0 con 0x7f71e01017f0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71d77fe640 1 -- 172.21.15.96:0/3470730103 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71dc010910 con 0x7f71e01017f0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71d77fe640 1 -- 172.21.15.96:0/3470730103 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc018a00 con 0x7f71e01017f0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01a1c10 con 0x7f71e01017f0 2024-03-18T15:48:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.894+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a21b0 con 0x7f71e01017f0 2024-03-18T15:48:14.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.897+0000 7f71eb372740 1 Processor -- start 2024-03-18T15:48:14.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.897+0000 7f71d77fe640 1 -- 172.21.15.96:0/3470730103 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f71dc01f020 con 0x7f71e01017f0 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 -- start start 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702cf3b0 0x5564702d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55647022e490 con 0x5564702ca240 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5564701d6c90 con 0x5564702c9850 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71eb372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55647022a530 con 0x5564702cf3b0 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55980/0 (socket says 172.21.15.96:55980) 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 -- 172.21.15.96:0/2741236092 learned_addr learned my addr 172.21.15.96:0/2741236092 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e5c47640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702cf3b0 0x5564702d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 -- 172.21.15.96:0/2741236092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702cf3b0 msgr2=0x5564702d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702cf3b0 0x5564702d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 -- 172.21.15.96:0/2741236092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 msgr2=0x5564702c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6448640 1 -- 172.21.15.96:0/2741236092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556470291500 con 0x5564702ca240 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71d77fe640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 0x7f71cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.898+0000 7f71e6c49640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 0x7f71cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71d77fe640 1 -- 172.21.15.96:0/3470730103 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f71dc0cec80 con 0x7f71e01017f0 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71e6448640 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0d2e20 tx=0x7f71dc097e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75686864dd9b9cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71d4ff9640 1 -- 172.21.15.96:0/2741236092 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc09b3c0 con 0x5564702ca240 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71e6c49640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 0x7f71cc0a4d40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005e30 tx=0x7f71d8032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71d4ff9640 1 -- 172.21.15.96:0/2741236092 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71dc0d2e50 con 0x5564702ca240 2024-03-18T15:48:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.899+0000 7f71d4ff9640 1 -- 172.21.15.96:0/2741236092 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc099730 con 0x5564702ca240 2024-03-18T15:48:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 -- 172.21.15.96:0/2741236092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 msgr2=0x5564702ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0d2e20 tx=0x7f71dc097e30 comp rx=0 tx=0).stop 2024-03-18T15:48:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 -- 172.21.15.96:0/2741236092 shutdown_connections 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702cf3b0 0x5564702d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702ced40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 --2- 172.21.15.96:0/2741236092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 -- 172.21.15.96:0/2741236092 >> 172.21.15.96:0/2741236092 conn(0x5564702c4dc0 msgr2=0x5564702c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 -- 172.21.15.96:0/2741236092 shutdown_connections 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.900+0000 7f71eb372740 1 -- 172.21.15.96:0/2741236092 wait complete. 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 -- start start 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 Processor -- start 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702ca240 0x5564702d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564702d2360 con 0x5564702cf3b0 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5564701ea230 con 0x5564702ca240 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564702d24e0 con 0x5564702c9850 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71eb372740 10 client.0 _ll_register_callbacks cb 0x556470226010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71e6c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71e6c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55994/0 (socket says 172.21.15.96:55994) 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71e6c49640 1 -- 172.21.15.96:0/2521648132 learned_addr learned my addr 172.21.15.96:0/2521648132 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71e6448640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.901+0000 7f71e5c47640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702ca240 0x5564702d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 -- 172.21.15.96:0/2521648132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702ca240 msgr2=0x5564702d42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702ca240 0x5564702d42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 -- 172.21.15.96:0/2521648132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 msgr2=0x5564702db310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 -- 172.21.15.96:0/2521648132 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564701f9100 con 0x5564702c9850 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71e6448640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d3d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0cc690 tx=0x7f71dc018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e9487e7587c2708 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0b4060 con 0x5564702c9850 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71dc0d2e50 con 0x5564702c9850 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f71dc010430 con 0x5564702c9850 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71dc09af50 con 0x5564702c9850 2024-03-18T15:48:14.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71eb372740 10 client.27439 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55646ff423f0 con 0x5564702c9850 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.902+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71dc0d2e50 con 0x5564702c9850 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f71dc010ac0 con 0x5564702c9850 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 10 client.27439 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 10 client.27439 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 20 client.27439 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564702df800 0x5564702e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 4 client.27439 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647023b9d0 con 0x5564702df800 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.903+0000 7f71e5c47640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564702df800 0x5564702e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.904+0000 7f71c9ffb640 10 client.27439 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.904+0000 7f71e5c47640 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564702df800 0x5564702e1bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b86d544841be40d3 server_cookie=5f087f055175f02b in_seq=0 out_seq=0 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.904+0000 7f71c9ffb640 1 -- 172.21.15.96:0/2521648132 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55647023b9d0 con 0x5564702df800 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.904+0000 7f71c9ffb640 10 client.27439 handle_command_reply: tid=0 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.904+0000 7f71eb372740 1 client.27439 shutdown 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 msgr2=0x5564702d3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d3d90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0cc690 tx=0x7f71dc018420 comp rx=0 tx=0).stop 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 shutdown_connections 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564702df800 0x5564702e1bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71c9ffb640 0 client.27439 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702ca240 0x5564702d42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 --2- 172.21.15.96:0/2521648132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d3d90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 >> 172.21.15.96:0/2521648132 conn(0x5564702c4dc0 msgr2=0x5564702c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 shutdown_connections 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 1 -- 172.21.15.96:0/2521648132 wait complete. 2024-03-18T15:48:14.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.905+0000 7f71eb372740 20 client.27439 trim_cache size 0 max 16384 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 Processor -- start 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 -- start start 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01aaf20 con 0x5564702cf3b0 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01acb80 con 0x5564702c9850 2024-03-18T15:48:14.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71eb372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ab620 con 0x5564702ca240 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71e6c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71e5c47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71e5c47640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39368/0 (socket says 172.21.15.96:39368) 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71e5c47640 1 -- 172.21.15.96:0/2752701165 learned_addr learned my addr 172.21.15.96:0/2752701165 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.918+0000 7f71e6448640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 -- 172.21.15.96:0/2752701165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 msgr2=0x5564702db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 -- 172.21.15.96:0/2752701165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 msgr2=0x5564702d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 -- 172.21.15.96:0/2752701165 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647000c7d0 con 0x5564702cf3b0 2024-03-18T15:48:14.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6448640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e5c47640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:14.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71e6c49640 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702c3750 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f71d8037040 tx=0x7f71d8041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36433fea1960edc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71cbfff640 1 -- 172.21.15.96:0/2752701165 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8039040 con 0x5564702cf3b0 2024-03-18T15:48:14.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71cbfff640 1 -- 172.21.15.96:0/2752701165 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d8004040 con 0x5564702cf3b0 2024-03-18T15:48:14.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.919+0000 7f71cbfff640 1 -- 172.21.15.96:0/2752701165 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d804d650 con 0x5564702cf3b0 2024-03-18T15:48:14.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- 172.21.15.96:0/2752701165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 msgr2=0x5564702c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702c3750 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f71d8037040 tx=0x7f71d8041000 comp rx=0 tx=0).stop 2024-03-18T15:48:14.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- 172.21.15.96:0/2752701165 shutdown_connections 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702cf3b0 0x5564702c3750 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702ca240 0x5564702db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 --2- 172.21.15.96:0/2752701165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702c9850 0x5564702d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- 172.21.15.96:0/2752701165 >> 172.21.15.96:0/2752701165 conn(0x5564702c4dc0 msgr2=0x5564702d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- 172.21.15.96:0/2752701165 shutdown_connections 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- 172.21.15.96:0/2752701165 wait complete. 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.920+0000 7f71eb372740 1 -- start start 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 Processor -- start 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 0x5564702d9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01ac4b0 con 0x5564702ca240 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01abd20 con 0x5564702cf3b0 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01a8f00 con 0x5564702c9850 2024-03-18T15:48:14.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71eb372740 10 client.0 _ll_register_callbacks cb 0x7f71dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56012/0 (socket says 172.21.15.96:56012) 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 -- 172.21.15.96:0/3048680232 learned_addr learned my addr 172.21.15.96:0/3048680232 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e6448640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e6c49640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 0x5564702d9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 -- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 msgr2=0x5564702d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 -- 172.21.15.96:0/3048680232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 msgr2=0x5564702d9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 0x5564702d9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 -- 172.21.15.96:0/3048680232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564702da470 con 0x5564702ca240 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e6c49640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 0x5564702d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71e5c47640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x5564702e22b0 tx=0x7f71d001f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c29ecd7985c63e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.921+0000 7f71c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:14.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0023020 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f71d0003ac0 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d8004040 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71d002e050 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 10 client.17619 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5564703459a0 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71d8004040 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f71d001e410 con 0x5564702ca240 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 10 client.17619 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 10 client.17619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 20 client.17619 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564703463a0 0x5564703487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 4 client.17619 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556470235d20 con 0x5564703463a0 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.922+0000 7f71e6448640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564703463a0 0x5564703487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:14.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.923+0000 7f71e6448640 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564703463a0 0x5564703487c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=79b76cd08bb5eadd server_cookie=69b7f518b19a1744 in_seq=0 out_seq=0 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.923+0000 7f71c9ffb640 10 client.17619 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71c9ffb640 1 -- 172.21.15.96:0/3048680232 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556470235d20 con 0x5564703463a0 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71c9ffb640 10 client.17619 handle_command_reply: tid=0 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 client.17619 shutdown 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 msgr2=0x5564702d8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x5564702e22b0 tx=0x7f71d001f830 comp rx=0 tx=0).stop 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 shutdown_connections 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x5564703463a0 0x5564703487c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5564702cf3b0 0x5564702d9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564702ca240 0x5564702d8ee0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 --2- 172.21.15.96:0/3048680232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564702c9850 0x5564702d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 >> 172.21.15.96:0/3048680232 conn(0x5564702c4dc0 msgr2=0x5564702c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71c9ffb640 0 client.17619 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 shutdown_connections 2024-03-18T15:48:14.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 1 -- 172.21.15.96:0/3048680232 wait complete. 2024-03-18T15:48:14.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.924+0000 7f71eb372740 20 client.17619 trim_cache size 0 max 16384 2024-03-18T15:48:14.931 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:14.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 msgr2=0x7f71cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 0x7f71cc0a4d40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005e30 tx=0x7f71d8032590 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 msgr2=0x7f71e0199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00e9c0 tx=0x7f71dc00ee80 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 shutdown_connections 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f71cc0a2880 0x7f71cc0a4d40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71e01033a0 0x7f71e01a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71e01029e0 0x7f71e019a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 --2- 172.21.15.96:0/3470730103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71e01017f0 0x7f71e0199d10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 >> 172.21.15.96:0/3470730103 conn(0x7f71e00fcd60 msgr2=0x7f71e0073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 shutdown_connections 2024-03-18T15:48:14.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:14.925+0000 7f71e86ac640 1 -- 172.21.15.96:0/3470730103 wait complete. 2024-03-18T15:48:14.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:14 smithi096 ceph-mon[31930]: pgmap v597: 97 pgs: 97 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-03-18T15:48:14.935 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:14 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2508485006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:14.937 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- 172.21.15.96:0/3710894057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 msgr2=0x7f5104108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 --2- 172.21.15.96:0/3710894057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f5104108b30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009ca0 tx=0x7f50fc02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- 172.21.15.96:0/3710894057 shutdown_connections 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 --2- 172.21.15.96:0/3710894057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f5104108b30 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 --2- 172.21.15.96:0/3710894057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 0x7f5104101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 --2- 172.21.15.96:0/3710894057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 0x7f5104100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- 172.21.15.96:0/3710894057 >> 172.21.15.96:0/3710894057 conn(0x7f51040fbc80 msgr2=0x7f51040fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- 172.21.15.96:0/3710894057 shutdown_connections 2024-03-18T15:48:15.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- 172.21.15.96:0/3710894057 wait complete. 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 Processor -- start 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.044+0000 7f510a28c640 1 -- start start 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 0x7f5104197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 0x7f5104198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c370 con 0x7f51041022a0 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c1f0 con 0x7f51041018e0 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f510a28c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c4f0 con 0x7f51041006f0 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56036/0 (socket says 172.21.15.96:56036) 2024-03-18T15:48:15.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 -- 172.21.15.96:0/557187082 learned_addr learned my addr 172.21.15.96:0/557187082 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5103fff640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 0x7f5104197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f51037fe640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 0x7f5104198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 -- 172.21.15.96:0/557187082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 msgr2=0x7f5104197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 0x7f5104197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 -- 172.21.15.96:0/557187082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 msgr2=0x7f5104198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 0x7f5104198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 -- 172.21.15.96:0/557187082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510419f6f0 con 0x7f51041022a0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f5108829640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f50fc037040 tx=0x7f50fc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.045+0000 7f51017fa640 1 -- 172.21.15.96:0/557187082 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc041020 con 0x7f51041022a0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.046+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510419f980 con 0x7f51041022a0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.046+0000 7f51017fa640 1 -- 172.21.15.96:0/557187082 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50fc034cd0 con 0x7f51041022a0 2024-03-18T15:48:15.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.046+0000 7f51017fa640 1 -- 172.21.15.96:0/557187082 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc039040 con 0x7f51041022a0 2024-03-18T15:48:15.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.046+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510419fe70 con 0x7f51041022a0 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f51017fa640 1 -- 172.21.15.96:0/557187082 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f50fc044020 con 0x7f51041022a0 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 Processor -- start 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 -- start start 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f915d850 0x5591f915dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 0x5591f9165840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591f90c2490 con 0x5591f915e240 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5591f906ac90 con 0x5591f91633b0 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f510ce72740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591f90be530 con 0x5591f915d850 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56048/0 (socket says 172.21.15.96:56048) 2024-03-18T15:48:15.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f51017fa640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 0x7f50ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f5103fff640 1 -- 172.21.15.96:0/1812791356 learned_addr learned my addr 172.21.15.96:0/1812791356 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f51037fe640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 0x5591f9165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.048+0000 7f51037fe640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f915d850 0x5591f915dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5108829640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 0x7f50ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 -- 172.21.15.96:0/1812791356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f915d850 msgr2=0x5591f915dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f915d850 0x5591f915dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 -- 172.21.15.96:0/1812791356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 msgr2=0x5591f9165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 0x5591f9165840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 -- 172.21.15.96:0/1812791356 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f9125500 con 0x5591f915e240 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f51017fa640 1 -- 172.21.15.96:0/557187082 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f50fc0f08b0 con 0x7f51041022a0 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f51037fe640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 0x5591f9165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5103fff640 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009ca0 tx=0x7f50f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d768f7e8fb9ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f50ea7fc640 1 -- 172.21.15.96:0/1812791356 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4004220 con 0x5591f915e240 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f5108829640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 0x7f50ec0a4cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f50fc08ffb0 tx=0x7f50fc08ffe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f50ea7fc640 1 -- 172.21.15.96:0/1812791356 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f40043c0 con 0x5591f915e240 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.049+0000 7f50ea7fc640 1 -- 172.21.15.96:0/1812791356 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4019890 con 0x5591f915e240 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- 172.21.15.96:0/1812791356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 msgr2=0x5591f9162d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009ca0 tx=0x7f50f4011810 comp rx=0 tx=0).stop 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- 172.21.15.96:0/1812791356 shutdown_connections 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f91633b0 0x5591f9165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f9162d40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 --2- 172.21.15.96:0/1812791356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f915d850 0x5591f915dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- 172.21.15.96:0/1812791356 >> 172.21.15.96:0/1812791356 conn(0x5591f9158dc0 msgr2=0x5591f91591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- 172.21.15.96:0/1812791356 shutdown_connections 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- 172.21.15.96:0/1812791356 wait complete. 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 -- start start 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.050+0000 7f510ce72740 1 Processor -- start 2024-03-18T15:48:15.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9167d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591f9166360 con 0x5591f915e240 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5591f907e230 con 0x5591f915d850 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591f91664e0 con 0x5591f91633b0 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 10 client.0 _ll_register_callbacks cb 0x5591f90ba010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56056/0 (socket says 172.21.15.96:56056) 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 -- 172.21.15.96:0/3756532191 learned_addr learned my addr 172.21.15.96:0/3756532191 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f51037fe640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f51037fe640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9167d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 -- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 msgr2=0x5591f916f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 -- 172.21.15.96:0/3756532191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 msgr2=0x5591f9167d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9167d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 -- 172.21.15.96:0/3756532191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f908d100 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f5103fff640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f50f401b040 tx=0x7f50f4005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32654704649e8bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f50e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4028070 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f40043c0 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f50f4005390 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f50f4034050 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 10 client.17631 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.051+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5591f8dd63f0 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f40043c0 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f50f4004880 con 0x5591f915e240 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 10 client.17631 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 10 client.17631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 20 client.17631 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91739b0 0x5591f9175e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 4 client.17631 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591f90cf9d0 con 0x5591f91739b0 2024-03-18T15:48:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f51037fe640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91739b0 0x5591f9175e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f51037fe640 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91739b0 0x5591f9175e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48087195a491a633 server_cookie=14a179be97fa0361 in_seq=0 out_seq=0 2024-03-18T15:48:15.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.052+0000 7f50e37fe640 10 client.17631 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f50e37fe640 1 -- 172.21.15.96:0/3756532191 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5591f90cf9d0 con 0x5591f91739b0 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f50e37fe640 10 client.17631 handle_command_reply: tid=0 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 client.17631 shutdown 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 msgr2=0x5591f91682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f50f401b040 tx=0x7f50f4005cd0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 shutdown_connections 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91739b0 0x5591f9175e10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f91682a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 --2- 172.21.15.96:0/3756532191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9167d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f50e37fe640 0 client.17631 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 >> 172.21.15.96:0/3756532191 conn(0x5591f9158dc0 msgr2=0x5591f9157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 shutdown_connections 2024-03-18T15:48:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 1 -- 172.21.15.96:0/3756532191 wait complete. 2024-03-18T15:48:15.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.056+0000 7f510ce72740 20 client.17631 trim_cache size 0 max 16384 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 Processor -- start 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 -- start start 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9157750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f9157c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51041a8d00 con 0x5591f915e240 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51041aa960 con 0x5591f915d850 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f510ce72740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51041a9400 con 0x5591f91633b0 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56066/0 (socket says 172.21.15.96:56066) 2024-03-18T15:48:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 -- 172.21.15.96:0/3112359299 learned_addr learned my addr 172.21.15.96:0/3112359299 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f51037fe640 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f51037fe640 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f9157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 -- 172.21.15.96:0/3112359299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 msgr2=0x5591f9157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f9157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 -- 172.21.15.96:0/3112359299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 msgr2=0x5591f9157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.069+0000 7f5103fff640 1 -- 172.21.15.96:0/3112359299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591f8ea07d0 con 0x5591f915e240 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f5103fff640 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009800 tx=0x7f50f4005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc405b2bf96bb80 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f50e9ffb640 1 -- 172.21.15.96:0/3112359299 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4028040 con 0x5591f915e240 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f50e9ffb640 1 -- 172.21.15.96:0/3112359299 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4031030 con 0x5591f915e240 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f50e9ffb640 1 -- 172.21.15.96:0/3112359299 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4019870 con 0x5591f915e240 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 -- 172.21.15.96:0/3112359299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 msgr2=0x5591f916f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009800 tx=0x7f50f4005050 comp rx=0 tx=0).stop 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 -- 172.21.15.96:0/3112359299 shutdown_connections 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f9157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916f170 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 --2- 172.21.15.96:0/3112359299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f9157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 -- 172.21.15.96:0/3112359299 >> 172.21.15.96:0/3112359299 conn(0x5591f9158dc0 msgr2=0x5591f91591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 -- 172.21.15.96:0/3112359299 shutdown_connections 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.070+0000 7f510ce72740 1 -- 172.21.15.96:0/3112359299 wait complete. 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 -- start start 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 Processor -- start 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f916cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51041aa290 con 0x5591f915e240 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f51041a9b00 con 0x5591f915d850 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51041a6ce0 con 0x5591f91633b0 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f510ce72740 10 client.0 _ll_register_callbacks cb 0x7f50f8009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f51037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f916cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f51037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f916cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43752/0 (socket says 172.21.15.96:43752) 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f5103fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56080/0 (socket says 172.21.15.96:56080) 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f51037fe640 1 -- 172.21.15.96:0/706797532 learned_addr learned my addr 172.21.15.96:0/706797532 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f51037fe640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f5103fff640 1 -- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 msgr2=0x5591f916e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.071+0000 7f5103fff640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f5103fff640 1 -- 172.21.15.96:0/706797532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 msgr2=0x5591f916cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f5103fff640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f916cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f5103fff640 1 -- 172.21.15.96:0/706797532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591f916e740 con 0x5591f915e240 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f5103fff640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004cd0 tx=0x7f50f4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241d014f480e762d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f403d030 con 0x5591f915e240 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f50f4019830 con 0x5591f915e240 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f4031030 con 0x5591f915e240 2024-03-18T15:48:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f50f4004e70 con 0x5591f915e240 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 10 client.17637 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5591f91da080 con 0x5591f915e240 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f4031030 con 0x5591f915e240 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f50f40035f0 con 0x5591f915e240 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 10 client.17637 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 20 client.17637 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91daa80 0x5591f91dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 4 client.17637 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.072+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591f915f1b0 con 0x5591f91daa80 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.073+0000 7f51037fe640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91daa80 0x5591f91dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.073+0000 7f51037fe640 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91daa80 0x5591f91dcea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4db87598db809f8c server_cookie=7c94280c5353c125 in_seq=0 out_seq=0 2024-03-18T15:48:15.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.073+0000 7f50e37fe640 10 client.17637 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f50e37fe640 1 -- 172.21.15.96:0/706797532 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591f915f1b0 con 0x5591f91daa80 2024-03-18T15:48:15.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f50e37fe640 10 client.17637 handle_command_reply: tid=0 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 client.17637 shutdown 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 msgr2=0x5591f916d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f50f4004cd0 tx=0x7f50f4019660 comp rx=0 tx=0).stop 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 shutdown_connections 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x5591f91daa80 0x5591f91dcea0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591f91633b0 0x5591f916e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591f915e240 0x5591f916d190 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f50e37fe640 0 client.17637 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 --2- 172.21.15.96:0/706797532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5591f915d850 0x5591f916cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 >> 172.21.15.96:0/706797532 conn(0x5591f9158dc0 msgr2=0x5591f915c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 shutdown_connections 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 1 -- 172.21.15.96:0/706797532 wait complete. 2024-03-18T15:48:15.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510ce72740 20 client.17637 trim_cache size 0 max 16384 2024-03-18T15:48:15.081 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 msgr2=0x7f50ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 0x7f50ec0a4cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f50fc08ffb0 tx=0x7f50fc08ffe0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 msgr2=0x7f510419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.076+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f50fc037040 tx=0x7f50fc032f40 comp rx=0 tx=0).stop 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 shutdown_connections 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f50ec0a2800 0x7f50ec0a4cc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51041022a0 0x7f510419f080 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51041018e0 0x7f5104198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 --2- 172.21.15.96:0/557187082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51041006f0 0x7f5104197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 >> 172.21.15.96:0/557187082 conn(0x7f51040fbc80 msgr2=0x7f51040fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 shutdown_connections 2024-03-18T15:48:15.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.077+0000 7f510a28c640 1 -- 172.21.15.96:0/557187082 wait complete. 2024-03-18T15:48:15.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:48:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:14 smithi137 ceph-mon[35868]: pgmap v597: 97 pgs: 97 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-03-18T15:48:15.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:14 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2508485006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:15.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:14 smithi193 ceph-mon[36702]: pgmap v597: 97 pgs: 97 active+clean; 283 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 359 op/s 2024-03-18T15:48:15.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:14 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2508485006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 -- 172.21.15.96:0/2103168132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874107570 msgr2=0x7f2874109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 --2- 172.21.15.96:0/2103168132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874107570 0x7f2874109960 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f286c009ca0 tx=0x7f286c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 -- 172.21.15.96:0/2103168132 shutdown_connections 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 --2- 172.21.15.96:0/2103168132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874107570 0x7f2874109960 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 --2- 172.21.15.96:0/2103168132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874104c40 0x7f2874107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 --2- 172.21.15.96:0/2103168132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874102310 0x7f2874104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 -- 172.21.15.96:0/2103168132 >> 172.21.15.96:0/2103168132 conn(0x7f28740fbc80 msgr2=0x7f28740fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 -- 172.21.15.96:0/2103168132 shutdown_connections 2024-03-18T15:48:15.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.210+0000 7f287943d640 1 -- 172.21.15.96:0/2103168132 wait complete. 2024-03-18T15:48:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 Processor -- start 2024-03-18T15:48:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 -- start start 2024-03-18T15:48:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874104c40 0x7f2874198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874107570 0x7f287419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d470 con 0x7f2874102310 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d2f0 con 0x7f2874104c40 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f287943d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f287410d5f0 con 0x7f2874107570 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56094/0 (socket says 172.21.15.96:56094) 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 -- 172.21.15.96:0/1023392449 learned_addr learned my addr 172.21.15.96:0/1023392449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f28727fc640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874104c40 0x7f2874198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f28737fe640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874107570 0x7f287419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 -- 172.21.15.96:0/1023392449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874107570 msgr2=0x7f287419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874107570 0x7f287419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 -- 172.21.15.96:0/1023392449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874104c40 msgr2=0x7f2874198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874104c40 0x7f2874198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.211+0000 7f2872ffd640 1 -- 172.21.15.96:0/1023392449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287419f710 con 0x7f2874102310 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9f0 tx=0x7f285c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f286bfff640 1 -- 172.21.15.96:0/1023392449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00cd80 con 0x7f2874102310 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287419fa00 con 0x7f2874102310 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f286bfff640 1 -- 172.21.15.96:0/1023392449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285c010910 con 0x7f2874102310 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f286bfff640 1 -- 172.21.15.96:0/1023392449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0188f0 con 0x7f2874102310 2024-03-18T15:48:15.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.212+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287419ffa0 con 0x7f2874102310 2024-03-18T15:48:15.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.214+0000 7f286bfff640 1 -- 172.21.15.96:0/1023392449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f285c010ac0 con 0x7f2874102310 2024-03-18T15:48:15.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.214+0000 7f287c022740 1 Processor -- start 2024-03-18T15:48:15.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.214+0000 7f287c022740 1 -- start start 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6af2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af83b0 0x555ac6afa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ac6a57490 con 0x555ac6af3240 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ac69ffc90 con 0x555ac6af2850 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f287c022740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555ac6a53530 con 0x555ac6af83b0 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56110/0 (socket says 172.21.15.96:56110) 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6af2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f2872ffd640 1 -- 172.21.15.96:0/4111249114 learned_addr learned my addr 172.21.15.96:0/4111249114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28727fc640 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af83b0 0x555ac6afa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f286bfff640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 0x7f28600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 -- 172.21.15.96:0/4111249114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af83b0 msgr2=0x555ac6afa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af83b0 0x555ac6afa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 -- 172.21.15.96:0/4111249114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 msgr2=0x555ac6af7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 -- 172.21.15.96:0/4111249114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac6aba500 con 0x555ac6af2850 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f2872ffd640 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f286bfff640 1 -- 172.21.15.96:0/1023392449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f285c0cee30 con 0x7f2874102310 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 0x7f28600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.215+0000 7f28737fe640 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6af2c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f286c02c630 tx=0x7f286c005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192dd351ba65396 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f28697fa640 1 -- 172.21.15.96:0/4111249114 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c039040 con 0x555ac6af2850 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f28737fe640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 0x7f28600a4cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f286c02c700 tx=0x7f286c045000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f28697fa640 1 -- 172.21.15.96:0/4111249114 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c003e50 con 0x555ac6af2850 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f28697fa640 1 -- 172.21.15.96:0/4111249114 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c04a440 con 0x555ac6af2850 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f287c022740 1 -- 172.21.15.96:0/4111249114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 msgr2=0x555ac6af2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.216+0000 7f287c022740 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6af2c70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f286c02c630 tx=0x7f286c005f70 comp rx=0 tx=0).stop 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- 172.21.15.96:0/4111249114 shutdown_connections 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af83b0 0x555ac6afa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- 172.21.15.96:0/4111249114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6af2c70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- 172.21.15.96:0/4111249114 >> 172.21.15.96:0/4111249114 conn(0x555ac6aeddc0 msgr2=0x555ac6aee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- 172.21.15.96:0/4111249114 shutdown_connections 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- 172.21.15.96:0/4111249114 wait complete. 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- start start 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 Processor -- start 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6afcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555ac6afb360 con 0x555ac6af83b0 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555ac6a13230 con 0x555ac6af3240 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555ac6afb4e0 con 0x555ac6af2850 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f287c022740 10 client.0 _ll_register_callbacks cb 0x555ac6a4f010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f28737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f28727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6afcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f28727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43778/0 (socket says 172.21.15.96:43778) 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f28737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56114/0 (socket says 172.21.15.96:56114) 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.217+0000 7f28727fc640 1 -- 172.21.15.96:0/828055959 learned_addr learned my addr 172.21.15.96:0/828055959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 -- 172.21.15.96:0/828055959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 msgr2=0x555ac6afcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6afcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 -- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 msgr2=0x555ac6afd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 -- 172.21.15.96:0/828055959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac6a22100 con 0x555ac6af83b0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28727fc640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f28737fe640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f286c037040 tx=0x7f286c004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b596bfe8a330fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c039070 con 0x555ac6af83b0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f286c003b70 con 0x555ac6af83b0 2024-03-18T15:48:15.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f286c003e50 con 0x555ac6af83b0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f286c032e90 con 0x555ac6af83b0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f287c022740 10 client.17649 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ac676b3f0 con 0x555ac6af83b0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c003e50 con 0x555ac6af83b0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.218+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f286c03f040 con 0x555ac6af83b0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 10 client.17649 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 10 client.17649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 20 client.17649 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b0abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 4 client.17649 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac6a649d0 con 0x555ac6b087c0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f2872ffd640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b0abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f285a7fc640 10 client.17649 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.219+0000 7f2872ffd640 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b0abb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e44e96b435f4557 server_cookie=2e4e3a436a6e14cf in_seq=0 out_seq=0 2024-03-18T15:48:15.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f285a7fc640 1 -- 172.21.15.96:0/828055959 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x555ac6a649d0 con 0x555ac6b087c0 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f285a7fc640 10 client.17649 handle_command_reply: tid=0 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 client.17649 shutdown 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 msgr2=0x555ac6b042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f286c037040 tx=0x7f286c004450 comp rx=0 tx=0).stop 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 shutdown_connections 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b042b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b0abb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af3240 0x555ac6afd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f285a7fc640 0 client.17649 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 --2- 172.21.15.96:0/828055959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6afcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 >> 172.21.15.96:0/828055959 conn(0x555ac6aeddc0 msgr2=0x555ac6aecfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 shutdown_connections 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 1 -- 172.21.15.96:0/828055959 wait complete. 2024-03-18T15:48:15.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.220+0000 7f287c022740 20 client.17649 trim_cache size 0 max 16384 2024-03-18T15:48:15.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.233+0000 7f287c022740 1 Processor -- start 2024-03-18T15:48:15.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.233+0000 7f287c022740 1 -- start start 2024-03-18T15:48:15.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6af47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 0x555ac6b04270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a8bf0 con 0x555ac6af3240 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28741aa850 con 0x555ac6af83b0 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f287c022740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a92f0 con 0x555ac6af2850 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f2872ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6af47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56120/0 (socket says 172.21.15.96:56120) 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 -- 172.21.15.96:0/4118198501 learned_addr learned my addr 172.21.15.96:0/4118198501 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28737fe640 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 0x555ac6b04270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 -- 172.21.15.96:0/4118198501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 msgr2=0x555ac6af47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6af47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 -- 172.21.15.96:0/4118198501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 msgr2=0x555ac6b04270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 0x555ac6b04270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 -- 172.21.15.96:0/4118198501 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac68357d0 con 0x555ac6af3240 2024-03-18T15:48:15.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28737fe640 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 0x555ac6b04270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:15.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f28727fc640 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2864012530 tx=0x7f2864005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765e5177faf5b659 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.234+0000 7f2868ff9640 1 -- 172.21.15.96:0/4118198501 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286400f480 con 0x555ac6af3240 2024-03-18T15:48:15.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f2868ff9640 1 -- 172.21.15.96:0/4118198501 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2864021070 con 0x555ac6af3240 2024-03-18T15:48:15.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f2868ff9640 1 -- 172.21.15.96:0/4118198501 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2864010900 con 0x555ac6af3240 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 -- 172.21.15.96:0/4118198501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 msgr2=0x555ac6af4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2864012530 tx=0x7f2864005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 -- 172.21.15.96:0/4118198501 shutdown_connections 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af83b0 0x555ac6b04270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af3240 0x555ac6af4ce0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 --2- 172.21.15.96:0/4118198501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af2850 0x555ac6af47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 -- 172.21.15.96:0/4118198501 >> 172.21.15.96:0/4118198501 conn(0x555ac6aeddc0 msgr2=0x555ac6aee1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 -- 172.21.15.96:0/4118198501 shutdown_connections 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.235+0000 7f287c022740 1 -- 172.21.15.96:0/4118198501 wait complete. 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 -- start start 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 Processor -- start 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6b01590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af3240 0x555ac6b01ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28741aa180 con 0x555ac6af83b0 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a99f0 con 0x555ac6af2850 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28741a6bd0 con 0x555ac6af3240 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f287c022740 10 client.0 _ll_register_callbacks cb 0x7f285c097af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56122/0 (socket says 172.21.15.96:56122) 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28727fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af3240 0x555ac6b01ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6b01590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 -- 172.21.15.96:0/1979916823 learned_addr learned my addr 172.21.15.96:0/1979916823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 -- 172.21.15.96:0/1979916823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af3240 msgr2=0x555ac6b01ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af3240 0x555ac6b01ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 -- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 msgr2=0x555ac6b01590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6b01590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 -- 172.21.15.96:0/1979916823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac6b030d0 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f28737fe640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f286c02cfd0 tx=0x7f286c02cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e5040dbd496bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f285a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c039070 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.236+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f286c003b70 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2864021070 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f286c06f050 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 10 client.17655 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ac6b043d0 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2864021070 con 0x555ac6af83b0 2024-03-18T15:48:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f286c066070 con 0x555ac6af83b0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 10 client.17655 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 20 client.17655 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b70f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 4 client.17655 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ac6a5ed20 con 0x555ac6b087c0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.237+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b70f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f2872ffd640 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b70f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50147c10a169a702 server_cookie=496a08207ad04ccc in_seq=0 out_seq=0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f285a7fc640 10 client.17655 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f285a7fc640 1 -- 172.21.15.96:0/1979916823 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ac6a5ed20 con 0x555ac6b087c0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f285a7fc640 10 client.17655 handle_command_reply: tid=0 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 client.17655 shutdown 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 msgr2=0x555ac6b02ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f286c02cfd0 tx=0x7f286c02cf60 comp rx=0 tx=0).stop 2024-03-18T15:48:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 shutdown_connections 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555ac6af83b0 0x555ac6b02ae0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555ac6af3240 0x555ac6b01ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x555ac6b087c0 0x555ac6b70f50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 --2- 172.21.15.96:0/1979916823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555ac6af2850 0x555ac6b01590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f285a7fc640 0 client.17655 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 >> 172.21.15.96:0/1979916823 conn(0x555ac6aeddc0 msgr2=0x555ac6afcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 shutdown_connections 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 1 -- 172.21.15.96:0/1979916823 wait complete. 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.238+0000 7f287c022740 20 client.17655 trim_cache size 0 max 16384 2024-03-18T15:48:15.245 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 msgr2=0x7f28600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 0x7f28600a4cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f286c02c700 tx=0x7f286c045000 comp rx=0 tx=0).stop 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 msgr2=0x7f2874197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f285c00e9f0 tx=0x7f285c00eeb0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 shutdown_connections 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f28600a2800 0x7f28600a4cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2874107570 0x7f287419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2874104c40 0x7f2874198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 --2- 172.21.15.96:0/1023392449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2874102310 0x7f2874197af0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 >> 172.21.15.96:0/1023392449 conn(0x7f28740fbc80 msgr2=0x7f2874102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 shutdown_connections 2024-03-18T15:48:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.239+0000 7f287943d640 1 -- 172.21.15.96:0/1023392449 wait complete. 2024-03-18T15:48:15.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.355+0000 7fd85a91b640 1 -- 172.21.15.96:0/2923595777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 msgr2=0x7fd854108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.355+0000 7fd85a91b640 1 --2- 172.21.15.96:0/2923595777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd854108b20 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009ca0 tx=0x7fd84c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.355+0000 7fd85a91b640 1 -- 172.21.15.96:0/2923595777 shutdown_connections 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- 172.21.15.96:0/2923595777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd854108b20 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- 172.21.15.96:0/2923595777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541018d0 0x7fd854101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- 172.21.15.96:0/2923595777 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541006e0 0x7fd854100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- 172.21.15.96:0/2923595777 >> 172.21.15.96:0/2923595777 conn(0x7fd8540fbc70 msgr2=0x7fd8540fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- 172.21.15.96:0/2923595777 shutdown_connections 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- 172.21.15.96:0/2923595777 wait complete. 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 Processor -- start 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- start start 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541006e0 0x7fd8540694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541018d0 0x7fd85419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd85410c360 con 0x7fd854102290 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd85410c1e0 con 0x7fd8541018d0 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.356+0000 7fd85a91b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd85410c4e0 con 0x7fd8541006e0 2024-03-18T15:48:15.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56138/0 (socket says 172.21.15.96:56138) 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd853fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541006e0 0x7fd8540694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 -- 172.21.15.96:0/1165103318 learned_addr learned my addr 172.21.15.96:0/1165103318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd8537fe640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541018d0 0x7fd85419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 -- 172.21.15.96:0/1165103318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541006e0 msgr2=0x7fd8540694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541006e0 0x7fd8540694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 -- 172.21.15.96:0/1165103318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541018d0 msgr2=0x7fd85419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541018d0 0x7fd85419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 -- 172.21.15.96:0/1165103318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85419f700 con 0x7fd854102290 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd858eb8640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd84c037040 tx=0x7fd84c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd8517fa640 1 -- 172.21.15.96:0/1165103318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c041020 con 0x7fd854102290 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85419f990 con 0x7fd854102290 2024-03-18T15:48:15.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd8517fa640 1 -- 172.21.15.96:0/1165103318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c034cd0 con 0x7fd854102290 2024-03-18T15:48:15.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd8517fa640 1 -- 172.21.15.96:0/1165103318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c039040 con 0x7fd854102290 2024-03-18T15:48:15.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.357+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85419fe80 con 0x7fd854102290 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.359+0000 7fd8517fa640 1 -- 172.21.15.96:0/1165103318 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd84c044020 con 0x7fd854102290 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 Processor -- start 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 -- start start 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff633490 con 0x55c6ff6d2320 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff5dbc90 con 0x55c6ff6ce7d0 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff62f530 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8517fa640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 0x7fd83c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56142/0 (socket says 172.21.15.96:56142) 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 -- 172.21.15.96:0/4196388126 learned_addr learned my addr 172.21.15.96:0/4196388126 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd853fff640 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd858eb8640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 0x7fd83c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 -- 172.21.15.96:0/4196388126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6cf1c0 msgr2=0x55c6ff6d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 -- 172.21.15.96:0/4196388126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 msgr2=0x55c6ff6cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 -- 172.21.15.96:0/4196388126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ff696500 con 0x55c6ff6d2320 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8517fa640 1 -- 172.21.15.96:0/1165103318 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd84c0f08b0 con 0x7fd854102290 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.360+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd8440117e0 tx=0x7fd8440097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcce2a261361c6c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd858eb8640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 0x7fd83c0a4cc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0c5490 tx=0x7fd84c0e7c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd83a7fc640 1 -- 172.21.15.96:0/4196388126 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844004030 con 0x55c6ff6d2320 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd83a7fc640 1 -- 172.21.15.96:0/4196388126 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84400fca0 con 0x55c6ff6d2320 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd83a7fc640 1 -- 172.21.15.96:0/4196388126 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440039f0 con 0x55c6ff6d2320 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4196388126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 msgr2=0x55c6ff6df190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd8440117e0 tx=0x7fd8440097d0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4196388126 shutdown_connections 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6df190 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4196388126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4196388126 >> 172.21.15.96:0/4196388126 conn(0x55c6ff6c9dc0 msgr2=0x55c6ff6ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4196388126 shutdown_connections 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.361+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4196388126 wait complete. 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 -- start start 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 Processor -- start 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff6dfb90 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff5ef230 con 0x55c6ff6ce7d0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c6ff6dfd10 con 0x55c6ff6d2320 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd85d5e4740 10 client.0 _ll_register_callbacks cb 0x55c6ff62b010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56154/0 (socket says 172.21.15.96:56154) 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 -- 172.21.15.96:0/4154044752 learned_addr learned my addr 172.21.15.96:0/4154044752 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 -- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 msgr2=0x55c6ff6e8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.362+0000 7fd853fff640 1 -- 172.21.15.96:0/4154044752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 msgr2=0x55c6ff6e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd853fff640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd853fff640 1 -- 172.21.15.96:0/4154044752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ff5fe100 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd853fff640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd848012700 tx=0x7fd848012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc9324e84e91539 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84800f8c0 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd84800fa60 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84400fca0 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd84800fc60 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 10 client.17667 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c6ff3473f0 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84400fca0 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fd848010550 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 10 client.17667 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 20 client.17667 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff6ed5a0 0x55c6ff6efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 4 client.17667 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.363+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6ff6409d0 con 0x55c6ff6ed5a0 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.364+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff6ed5a0 0x55c6ff6efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.364+0000 7fd8537fe640 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff6ed5a0 0x55c6ff6efa20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3214cb5f1e4659b server_cookie=69d419fa209c2510 in_seq=0 out_seq=0 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.364+0000 7fd8337fe640 10 client.17667 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd8337fe640 1 -- 172.21.15.96:0/4154044752 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55c6ff6409d0 con 0x55c6ff6ed5a0 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd8337fe640 10 client.17667 handle_command_reply: tid=0 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 client.17667 shutdown 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 msgr2=0x55c6ff6e1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd848012700 tx=0x7fd848012bc0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 shutdown_connections 2024-03-18T15:48:15.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff6ed5a0 0x55c6ff6efa20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/4154044752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 >> 172.21.15.96:0/4154044752 conn(0x55c6ff6c9dc0 msgr2=0x55c6ff6c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.365+0000 7fd8337fe640 0 client.17667 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.366+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 shutdown_connections 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.366+0000 7fd85d5e4740 1 -- 172.21.15.96:0/4154044752 wait complete. 2024-03-18T15:48:15.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.366+0000 7fd85d5e4740 20 client.17667 trim_cache size 0 max 16384 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 Processor -- start 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 -- start start 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a6ac0 con 0x55c6ff6d2320 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a8720 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a71c0 con 0x55c6ff6ce7d0 2024-03-18T15:48:15.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56168/0 (socket says 172.21.15.96:56168) 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 -- 172.21.15.96:0/2057954297 learned_addr learned my addr 172.21.15.96:0/2057954297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd853fff640 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 -- 172.21.15.96:0/2057954297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 msgr2=0x55c6ff6e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 -- 172.21.15.96:0/2057954297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 msgr2=0x55c6ff6e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.379+0000 7fd8537fe640 1 -- 172.21.15.96:0/2057954297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6ff4117d0 con 0x55c6ff6d2320 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd8537fe640 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd844009b40 tx=0x7fd844009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f61eb1e4da04cd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd839ffb640 1 -- 172.21.15.96:0/2057954297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844011110 con 0x55c6ff6d2320 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd839ffb640 1 -- 172.21.15.96:0/2057954297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8440112b0 con 0x55c6ff6d2320 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd839ffb640 1 -- 172.21.15.96:0/2057954297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84400f790 con 0x55c6ff6d2320 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 -- 172.21.15.96:0/2057954297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 msgr2=0x55c6ff6e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fd844009b40 tx=0x7fd844009b70 comp rx=0 tx=0).stop 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 -- 172.21.15.96:0/2057954297 shutdown_connections 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e8b00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/2057954297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 -- 172.21.15.96:0/2057954297 >> 172.21.15.96:0/2057954297 conn(0x55c6ff6c9dc0 msgr2=0x55c6ff6dbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 -- 172.21.15.96:0/2057954297 shutdown_connections 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.380+0000 7fd85d5e4740 1 -- 172.21.15.96:0/2057954297 wait complete. 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 -- start start 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 Processor -- start 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a8050 con 0x55c6ff6d2320 2024-03-18T15:48:15.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a78c0 con 0x55c6ff6cf1c0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8541a4aa0 con 0x55c6ff6ce7d0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd85d5e4740 10 client.0 _ll_register_callbacks cb 0x7fd848009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd8537fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56176/0 (socket says 172.21.15.96:56176) 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd8537fe640 1 -- 172.21.15.96:0/720866375 learned_addr learned my addr 172.21.15.96:0/720866375 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd853fff640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.381+0000 7fd8537fe640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 -- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 msgr2=0x55c6ff6e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 -- 172.21.15.96:0/720866375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 msgr2=0x55c6ff6e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 -- 172.21.15.96:0/720866375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6ff6e7950 con 0x55c6ff6d2320 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd853fff640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8537fe640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd8440058c0 tx=0x7fd844000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64c593043b9684c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:15.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844005180 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd84402f450 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8440112b0 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd84402f650 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 10 client.17673 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c6ff6e8c50 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8440112b0 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fd844003a60 con 0x55c6ff6d2320 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 10 client.17673 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 20 client.17673 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff753ec0 0x55c6ff7562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 4 client.17673 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.382+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6ff63ad20 con 0x55c6ff753ec0 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd853fff640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff753ec0 0x55c6ff7562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:15.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd8337fe640 10 client.17673 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd853fff640 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff753ec0 0x55c6ff7562e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1186f7f1a1e493f0 server_cookie=5ffefa6f39dcdbcc in_seq=0 out_seq=0 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd8337fe640 1 -- 172.21.15.96:0/720866375 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6ff63ad20 con 0x55c6ff753ec0 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd8337fe640 10 client.17673 handle_command_reply: tid=0 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 client.17673 shutdown 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 msgr2=0x55c6ff6e7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fd8440058c0 tx=0x7fd844000bc0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 shutdown_connections 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55c6ff753ec0 0x55c6ff7562e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6ff6d2320 0x55c6ff6e7360 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd8337fe640 0 client.17673 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6ff6cf1c0 0x55c6ff6e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 --2- 172.21.15.96:0/720866375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6ff6ce7d0 0x55c6ff6e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 >> 172.21.15.96:0/720866375 conn(0x55c6ff6c9dc0 msgr2=0x55c6ff6e0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 shutdown_connections 2024-03-18T15:48:15.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.383+0000 7fd85d5e4740 1 -- 172.21.15.96:0/720866375 wait complete. 2024-03-18T15:48:15.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85d5e4740 20 client.17673 trim_cache size 0 max 16384 2024-03-18T15:48:15.391 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 msgr2=0x7fd83c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 0x7fd83c0a4cc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0c5490 tx=0x7fd84c0e7c70 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 msgr2=0x7fd85419ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd84c037040 tx=0x7fd84c032f40 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 shutdown_connections 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd83c0a2800 0x7fd83c0a4cc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd854102290 0x7fd85419ee40 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd8541018d0 0x7fd85419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 --2- 172.21.15.96:0/1165103318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8541006e0 0x7fd8540694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 >> 172.21.15.96:0/1165103318 conn(0x7fd8540fbc70 msgr2=0x7fd854104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 shutdown_connections 2024-03-18T15:48:15.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:15.384+0000 7fd85a91b640 1 -- 172.21.15.96:0/1165103318 wait complete. 2024-03-18T15:48:16.396 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-18T15:48:16.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:48:16.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-18T15:48:16.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 -- 172.21.15.96:0/1326902432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1017f0 msgr2=0x7fe1dc101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 --2- 172.21.15.96:0/1326902432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc101c10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009900 tx=0x7fe1cc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 -- 172.21.15.96:0/1326902432 shutdown_connections 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 --2- 172.21.15.96:0/1326902432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 --2- 172.21.15.96:0/1326902432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 0x7fe1dc102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 --2- 172.21.15.96:0/1326902432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc101c10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 -- 172.21.15.96:0/1326902432 >> 172.21.15.96:0/1326902432 conn(0x7fe1dc0fcd80 msgr2=0x7fe1dc0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 -- 172.21.15.96:0/1326902432 shutdown_connections 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.507+0000 7fe1e226e640 1 -- 172.21.15.96:0/1326902432 wait complete. 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 Processor -- start 2024-03-18T15:48:16.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 -- start start 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 0x7fe1dc1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc10d460 con 0x7fe1dc1033a0 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc10d2e0 con 0x7fe1dc1017f0 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e226e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc10d5e0 con 0x7fe1dc1029e0 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56184/0 (socket says 172.21.15.96:56184) 2024-03-18T15:48:16.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 -- 172.21.15.96:0/207618615 learned_addr learned my addr 172.21.15.96:0/207618615 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 0x7fe1dc1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 -- 172.21.15.96:0/207618615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 msgr2=0x7fe1dc1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 0x7fe1dc1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 -- 172.21.15.96:0/207618615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1017f0 msgr2=0x7fe1dc197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.508+0000 7fe1e080b640 1 -- 172.21.15.96:0/207618615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc19f7a0 con 0x7fe1dc1033a0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1e080b640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009fd0 tx=0x7fe1d400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1d97fa640 1 -- 172.21.15.96:0/207618615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4017070 con 0x7fe1dc1033a0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc19fa90 con 0x7fe1dc1033a0 2024-03-18T15:48:16.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1d97fa640 1 -- 172.21.15.96:0/207618615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d4003e50 con 0x7fe1dc1033a0 2024-03-18T15:48:16.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1d97fa640 1 -- 172.21.15.96:0/207618615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40106c0 con 0x7fe1dc1033a0 2024-03-18T15:48:16.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.509+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc19ffe0 con 0x7fe1dc1033a0 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1d97fa640 1 -- 172.21.15.96:0/207618615 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe1d4003970 con 0x7fe1dc1033a0 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 Processor -- start 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 -- start start 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1d97fa640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 0x7fe1c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a4a173d0d0 con 0x55a4a17d8990 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4a16e5c70 con 0x55a4a17de4f0 2024-03-18T15:48:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.511+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a4a1739170 con 0x55a4a17d9380 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 0x7fe1c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56190/0 (socket says 172.21.15.96:56190) 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 -- 172.21.15.96:0/4083674272 learned_addr learned my addr 172.21.15.96:0/4083674272 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1e080b640 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1d97fa640 1 -- 172.21.15.96:0/207618615 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe1d40ce790 con 0x7fe1dc1033a0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 -- 172.21.15.96:0/4083674272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 msgr2=0x55a4a17dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 -- 172.21.15.96:0/4083674272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 msgr2=0x55a4a17e0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17e0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 -- 172.21.15.96:0/4083674272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a15197e0 con 0x55a4a17d8990 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 0x7fe1c40a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009710 tx=0x7fe1d0009ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009900 tx=0x7fe1cc0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de54acf91901212b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1c27fc640 1 -- 172.21.15.96:0/4083674272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc005d50 con 0x55a4a17d8990 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1c27fc640 1 -- 172.21.15.96:0/4083674272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1cc02cb80 con 0x55a4a17d8990 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.512+0000 7fe1c27fc640 1 -- 172.21.15.96:0/4083674272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc004920 con 0x55a4a17d8990 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- 172.21.15.96:0/4083674272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 msgr2=0x55a4a17d8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009900 tx=0x7fe1cc0095e0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- 172.21.15.96:0/4083674272 shutdown_connections 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17e0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/4083674272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d8990 0x55a4a17d8db0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- 172.21.15.96:0/4083674272 >> 172.21.15.96:0/4083674272 conn(0x55a4a17a0c50 msgr2=0x55a4a17d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- 172.21.15.96:0/4083674272 shutdown_connections 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- 172.21.15.96:0/4083674272 wait complete. 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.513+0000 7fe1e4e57740 1 -- start start 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 Processor -- start 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a4a17e1440 con 0x55a4a17d9380 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4a16f90d0 con 0x55a4a17de4f0 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a4a17e15c0 con 0x55a4a17d8990 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e4e57740 10 client.0 _ll_register_callbacks cb 0x55a4a16c2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56196/0 (socket says 172.21.15.96:56196) 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 -- 172.21.15.96:0/1599640176 learned_addr learned my addr 172.21.15.96:0/1599640176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1e080b640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 -- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 msgr2=0x55a4a17e2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 -- 172.21.15.96:0/1599640176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 msgr2=0x55a4a17ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 -- 172.21.15.96:0/1599640176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a1579f70 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0004260 tx=0x7fe1d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85213c76fd8eb380 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.514+0000 7fe1bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00038c0 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe1d0003a60 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1cc02cb80 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe1d0030030 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 10 client.17685 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a4a1411170 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1cc02cb80 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fe1d0020050 con 0x55a4a17d9380 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 10 client.17685 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 20 client.17685 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a17eeab0 0x55a4a17f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 4 client.17685 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4a174a610 con 0x55a4a17eeab0 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.515+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a17eeab0 0x55a4a17f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.516+0000 7fe1bb7fe640 10 client.17685 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.516+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a17eeab0 0x55a4a17f0ef0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a21d4da2db354bf3 server_cookie=8f120832156a52c0 in_seq=0 out_seq=0 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/1599640176 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55a4a174a610 con 0x55a4a17eeab0 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1bb7fe640 10 client.17685 handle_command_reply: tid=0 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 client.17685 shutdown 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 msgr2=0x55a4a17e33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0004260 tx=0x7fe1d0004730 comp rx=0 tx=0).stop 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 shutdown_connections 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a17eeab0 0x55a4a17f0ef0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17d9380 0x55a4a17e33e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1bb7fe640 0 client.17685 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/1599640176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 >> 172.21.15.96:0/1599640176 conn(0x55a4a17a0c50 msgr2=0x55a4a17d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 shutdown_connections 2024-03-18T15:48:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 1 -- 172.21.15.96:0/1599640176 wait complete. 2024-03-18T15:48:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.517+0000 7fe1e4e57740 20 client.17685 trim_cache size 0 max 16384 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.536+0000 7fe1e4e57740 1 Processor -- start 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.536+0000 7fe1e4e57740 1 -- start start 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 0x55a4a17d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1a8d00 con 0x55a4a17de4f0 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1aa960 con 0x55a4a17d8990 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1a9400 con 0x55a4a17d9380 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56206/0 (socket says 172.21.15.96:56206) 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 -- 172.21.15.96:0/629990967 learned_addr learned my addr 172.21.15.96:0/629990967 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1db7fe640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 0x55a4a17d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 -- 172.21.15.96:0/629990967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 msgr2=0x55a4a17d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 -- 172.21.15.96:0/629990967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 msgr2=0x55a4a17d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 0x55a4a17d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 -- 172.21.15.96:0/629990967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a17e1f50 con 0x55a4a17de4f0 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 0x55a4a17d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:16.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1e080b640 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005c40 tx=0x7fe1d40bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77c3bf3fc99e985 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1c1ffb640 1 -- 172.21.15.96:0/629990967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4020040 con 0x55a4a17de4f0 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1c1ffb640 1 -- 172.21.15.96:0/629990967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d40bb0b0 con 0x55a4a17de4f0 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.537+0000 7fe1c1ffb640 1 -- 172.21.15.96:0/629990967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40040b0 con 0x55a4a17de4f0 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- 172.21.15.96:0/629990967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 msgr2=0x55a4a17ea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4005c40 tx=0x7fe1d40bb040 comp rx=0 tx=0).stop 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- 172.21.15.96:0/629990967 shutdown_connections 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17ea3e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d9380 0x55a4a17d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/629990967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d8990 0x55a4a17d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- 172.21.15.96:0/629990967 >> 172.21.15.96:0/629990967 conn(0x55a4a17a0c50 msgr2=0x55a4a17d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- 172.21.15.96:0/629990967 shutdown_connections 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- 172.21.15.96:0/629990967 wait complete. 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 -- start start 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.538+0000 7fe1e4e57740 1 Processor -- start 2024-03-18T15:48:16.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d9380 0x55a4a17ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1aa290 con 0x55a4a17de4f0 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1a9b00 con 0x55a4a17d9380 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc1a6ce0 con 0x55a4a17d8990 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 10 client.0 _ll_register_callbacks cb 0x7fe1d0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1db7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d9380 0x55a4a17ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56222/0 (socket says 172.21.15.96:56222) 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 -- 172.21.15.96:0/3912648494 learned_addr learned my addr 172.21.15.96:0/3912648494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 -- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 msgr2=0x55a4a17e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 -- 172.21.15.96:0/3912648494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d9380 msgr2=0x55a4a17ea420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d9380 0x55a4a17ea420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 -- 172.21.15.96:0/3912648494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a17e8db0 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e080b640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe1d409aeb0 tx=0x7fe1d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509c2ef4392dc7d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4020040 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe1d4097820 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.539+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1d40bb0b0 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe1d409b1b0 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 10 client.17691 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a4a1855a40 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d40bb0b0 con 0x55a4a17de4f0 2024-03-18T15:48:16.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fe1d409a600 con 0x55a4a17de4f0 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 10 client.17691 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 20 client.17691 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a1856460 0x55a4a1858880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 4 client.17691 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a4a1744960 con 0x55a4a1856460 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.540+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a1856460 0x55a4a1858880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.541+0000 7fe1dbfff640 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a1856460 0x55a4a1858880 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66a8a815faf684d server_cookie=7e263a7986fadd43 in_seq=0 out_seq=0 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.541+0000 7fe1bb7fe640 10 client.17691 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1bb7fe640 1 -- 172.21.15.96:0/3912648494 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a4a1744960 con 0x55a4a1856460 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1bb7fe640 10 client.17691 handle_command_reply: tid=0 2024-03-18T15:48:16.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 client.17691 shutdown 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 msgr2=0x55a4a17e8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe1d409aeb0 tx=0x7fe1d400bf10 comp rx=0 tx=0).stop 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 shutdown_connections 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x55a4a1856460 0x55a4a1858880 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a4a17de4f0 0x55a4a17e8580 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a17d9380 0x55a4a17ea420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 --2- 172.21.15.96:0/3912648494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4a17d8990 0x55a4a17e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 >> 172.21.15.96:0/3912648494 conn(0x55a4a17a0c50 msgr2=0x55a4a17d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1bb7fe640 0 client.17691 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 shutdown_connections 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 1 -- 172.21.15.96:0/3912648494 wait complete. 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.542+0000 7fe1e4e57740 20 client.17691 trim_cache size 0 max 16384 2024-03-18T15:48:16.549 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:16.550 INFO:teuthology.orchestra.run.smithi096.stdout: "return_code": 0, 2024-03-18T15:48:16.550 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_tag": "abfc4d20-82f7-40f2-b3c9-d7337688ca3a", 2024-03-18T15:48:16.550 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": "asynchronous" 2024-03-18T15:48:16.550 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 msgr2=0x7fe1c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 0x7fe1c40a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009710 tx=0x7fe1d0009ad0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 msgr2=0x7fe1dc19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009fd0 tx=0x7fe1d400b260 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 shutdown_connections 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe1c40a2800 0x7fe1c40a4cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1dc1033a0 0x7fe1dc19f130 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1dc1029e0 0x7fe1dc1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 --2- 172.21.15.96:0/207618615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1dc1017f0 0x7fe1dc197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 >> 172.21.15.96:0/207618615 conn(0x7fe1dc0fcd80 msgr2=0x7fe1dc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 shutdown_connections 2024-03-18T15:48:16.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.543+0000 7fe1e226e640 1 -- 172.21.15.96:0/207618615 wait complete. 2024-03-18T15:48:16.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:48:16.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 -- 172.21.15.96:0/2504999014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe284103270 msgr2=0x7fe284109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/2504999014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe284103270 0x7fe284109b00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009ca0 tx=0x7fe27c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 -- 172.21.15.96:0/2504999014 shutdown_connections 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/2504999014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe284103270 0x7fe284109b00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/2504999014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 0x7fe284102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/2504999014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2841016c0 0x7fe284101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 -- 172.21.15.96:0/2504999014 >> 172.21.15.96:0/2504999014 conn(0x7fe2840fcc50 msgr2=0x7fe2840ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 -- 172.21.15.96:0/2504999014 shutdown_connections 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.661+0000 7fe28ab9d640 1 -- 172.21.15.96:0/2504999014 wait complete. 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 Processor -- start 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 -- start start 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 0x7fe284197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe284103270 0x7fe28419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410d320 con 0x7fe2841016c0 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410d1a0 con 0x7fe284103270 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28ab9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe28410d4a0 con 0x7fe2841028b0 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56238/0 (socket says 172.21.15.96:56238) 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 -- 172.21.15.96:0/3254849337 learned_addr learned my addr 172.21.15.96:0/3254849337 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28939a640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 0x7fe284197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe28a39c640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe284103270 0x7fe28419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 -- 172.21.15.96:0/3254849337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 msgr2=0x7fe284197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 0x7fe284197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 -- 172.21.15.96:0/3254849337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe284103270 msgr2=0x7fe28419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe284103270 0x7fe28419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 -- 172.21.15.96:0/3254849337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28419f550 con 0x7fe2841016c0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.662+0000 7fe289b9b640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe28000e9c0 tx=0x7fe28000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.663+0000 7fe27affd640 1 -- 172.21.15.96:0/3254849337 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28000cd80 con 0x7fe2841016c0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.663+0000 7fe27affd640 1 -- 172.21.15.96:0/3254849337 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2800035f0 con 0x7fe2841016c0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.663+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28419f840 con 0x7fe2841016c0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.663+0000 7fe27affd640 1 -- 172.21.15.96:0/3254849337 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280010690 con 0x7fe2841016c0 2024-03-18T15:48:16.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.663+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28419fd90 con 0x7fe2841016c0 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.665+0000 7fe27affd640 1 -- 172.21.15.96:0/3254849337 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fe2800037a0 con 0x7fe2841016c0 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.665+0000 7fe28e9ed740 1 Processor -- start 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.665+0000 7fe28e9ed740 1 -- start start 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557073fe9500 con 0x557074084d30 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557073f91be0 con 0x557074089ea0 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557073fe55a0 con 0x557074084340 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe27affd640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 0x7fe2700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28a39c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56242/0 (socket says 172.21.15.96:56242) 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 -- 172.21.15.96:0/2455271039 learned_addr learned my addr 172.21.15.96:0/2455271039 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28a39c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:39638/0 (socket says 172.21.15.96:39638) 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28939a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28a39c640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 0x7fe2700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe27affd640 1 -- 172.21.15.96:0/3254849337 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe2800cee40 con 0x7fe2841016c0 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 -- 172.21.15.96:0/2455271039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 msgr2=0x557074084760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 -- 172.21.15.96:0/2455271039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 msgr2=0x55707408c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 -- 172.21.15.96:0/2455271039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557073fa54b0 con 0x557074084d30 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28939a640 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28a39c640 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe289b9b640 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe280098180 tx=0x7fe28006e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fcfce48e8b5904 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe28a39c640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 0x7fe2700a4cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005e30 tx=0x7fe27c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.666+0000 7fe26ffff640 1 -- 172.21.15.96:0/2455271039 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28009beb0 con 0x557074084d30 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe26ffff640 1 -- 172.21.15.96:0/2455271039 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28009b7b0 con 0x557074084d30 2024-03-18T15:48:16.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe26ffff640 1 -- 172.21.15.96:0/2455271039 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280099b40 con 0x557074084d30 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 -- 172.21.15.96:0/2455271039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 msgr2=0x557074089830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe280098180 tx=0x7fe28006e4e0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 -- 172.21.15.96:0/2455271039 shutdown_connections 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x557074089830 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/2455271039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084340 0x557074084760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 -- 172.21.15.96:0/2455271039 >> 172.21.15.96:0/2455271039 conn(0x55707407f8b0 msgr2=0x55707407fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 -- 172.21.15.96:0/2455271039 shutdown_connections 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.667+0000 7fe28e9ed740 1 -- 172.21.15.96:0/2455271039 wait complete. 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 -- start start 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 Processor -- start 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x557074095dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55707408ce50 con 0x557074084d30 2024-03-18T15:48:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557073fa5160 con 0x557074084340 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55707408cfd0 con 0x557074089ea0 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28e9ed740 10 client.0 _ll_register_callbacks cb 0x557073f89150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28939a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43970/0 (socket says 172.21.15.96:43970) 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28939a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56244/0 (socket says 172.21.15.96:56244) 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 -- 172.21.15.96:0/1978378945 learned_addr learned my addr 172.21.15.96:0/1978378945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe28a39c640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x557074095dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 -- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 msgr2=0x557074095dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x557074095dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 -- 172.21.15.96:0/1978378945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 msgr2=0x55707408ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.668+0000 7fe289b9b640 1 -- 172.21.15.96:0/1978378945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557073dc77d0 con 0x557074084340 2024-03-18T15:48:16.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28939a640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe28009b4b0 tx=0x7fe280097410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8641cfbe7c9155b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28009beb0 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe280003b30 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28009b7b0 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe280086070 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 10 client.27506 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557073cfd3f0 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28009b7b0 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fe28007a0c0 con 0x557074084340 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 10 client.27506 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 20 client.27506 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557074099f30 0x55707409c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 4 client.27506 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:16.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.669+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557073ff6a40 con 0x557074099f30 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.670+0000 7fe28939a640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557074099f30 0x55707409c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.670+0000 7fe26d7fa640 10 client.27506 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.670+0000 7fe28939a640 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557074099f30 0x55707409c320 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102150ccb5fa52cf server_cookie=8ba893f6227401e8 in_seq=0 out_seq=0 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1978378945 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557073ff6a40 con 0x557074099f30 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe26d7fa640 10 client.27506 handle_command_reply: tid=0 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 client.27506 shutdown 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 msgr2=0x55707408e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe28009b4b0 tx=0x7fe280097410 comp rx=0 tx=0).stop 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 shutdown_connections 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x557074095dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x557074099f30 0x55707409c320 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084d30 0x55707408ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe26d7fa640 0 client.27506 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1978378945 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084340 0x55707408e850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 >> 172.21.15.96:0/1978378945 conn(0x55707407f8b0 msgr2=0x55707407ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 shutdown_connections 2024-03-18T15:48:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1978378945 wait complete. 2024-03-18T15:48:16.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.671+0000 7fe28e9ed740 20 client.27506 trim_cache size 0 max 16384 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 Processor -- start 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 -- start start 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 0x557074095d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841a8c10 con 0x557074084340 2024-03-18T15:48:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841aa870 con 0x557074089ea0 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841a9310 con 0x557074084d30 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56252/0 (socket says 172.21.15.96:56252) 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 -- 172.21.15.96:0/1823076992 learned_addr learned my addr 172.21.15.96:0/1823076992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28a39c640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28939a640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 0x557074095d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 -- 172.21.15.96:0/1823076992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 msgr2=0x557074095d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 0x557074095d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 -- 172.21.15.96:0/1823076992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 msgr2=0x55707408e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe289b9b640 1 -- 172.21.15.96:0/1823076992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557073cbd0f0 con 0x557074084340 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.690+0000 7fe28939a640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 0x557074095d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe28009b280 tx=0x7fe28009b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d5486a9bb60c15 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe26f7fe640 1 -- 172.21.15.96:0/1823076992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28009beb0 con 0x557074084340 2024-03-18T15:48:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe26f7fe640 1 -- 172.21.15.96:0/1823076992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28009b7b0 con 0x557074084340 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe26f7fe640 1 -- 172.21.15.96:0/1823076992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28007a020 con 0x557074084340 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1823076992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 msgr2=0x55707408e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.691+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe28009b280 tx=0x7fe28009b6b0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1823076992 shutdown_connections 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074089ea0 0x55707408e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074084d30 0x557074095d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1823076992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x55707408e460 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1823076992 >> 172.21.15.96:0/1823076992 conn(0x55707407f8b0 msgr2=0x55707407a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1823076992 shutdown_connections 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1823076992 wait complete. 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- start start 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 Processor -- start 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 0x557074093180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x5570740fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841aa1a0 con 0x557074084340 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841a9a10 con 0x557074084d30 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2841a6bf0 con 0x557074089ea0 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28e9ed740 10 client.0 _ll_register_callbacks cb 0x7fe27400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe289b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56258/0 (socket says 172.21.15.96:56258) 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe289b9b640 1 -- 172.21.15.96:0/1368537128 learned_addr learned my addr 172.21.15.96:0/1368537128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.692+0000 7fe28939a640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 0x557074093180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe28a39c640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x5570740fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 -- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 msgr2=0x5570740fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x5570740fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 -- 172.21.15.96:0/1368537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 msgr2=0x557074093180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 0x557074093180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 -- 172.21.15.96:0/1368537128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557073fb09d0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe28939a640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 0x557074093180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe289b9b640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe280005eb0 tx=0x7fe2800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487928f7f2fe10a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280073060 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fe28007a0a0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28009b7b0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe28009beb0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe28e9ed740 10 client.17709 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557073fb09d0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe28009b7b0 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.693+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7fe28009ac90 con 0x557074084340 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 10 client.17709 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:16.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 20 client.17709 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5570740ffcb0 0x5570741020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 4 client.17709 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557073ff0d90 con 0x5570740ffcb0 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28939a640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5570740ffcb0 0x5570741020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28939a640 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5570740ffcb0 0x5570741020d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb025fa2e5dff3e server_cookie=2632671355512ef8 in_seq=0 out_seq=0 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe26d7fa640 10 client.17709 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe26d7fa640 1 -- 172.21.15.96:0/1368537128 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557073ff0d90 con 0x5570740ffcb0 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe26d7fa640 10 client.17709 handle_command_reply: tid=0 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 1 client.17709 shutdown 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 msgr2=0x557074092c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.694+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe280005eb0 tx=0x7fe2800046c0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 shutdown_connections 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557074089ea0 0x5570740fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557074084d30 0x557074093180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5570740ffcb0 0x5570741020d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe26d7fa640 0 client.17709 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 --2- 172.21.15.96:0/1368537128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557074084340 0x557074092c40 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 >> 172.21.15.96:0/1368537128 conn(0x55707407f8b0 msgr2=0x55707407e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 shutdown_connections 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 1 -- 172.21.15.96:0/1368537128 wait complete. 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28e9ed740 20 client.17709 trim_cache size 0 max 16384 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-18T15:48:16.702 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": { 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "abfc4d20-82f7-40f2-b3c9-d7337688ca3a": { 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/", 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tag": "abfc4d20-82f7-40f2-b3c9-d7337688ca3a", 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: "options": "recursive,force" 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:16.703 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 msgr2=0x7fe2700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 0x7fe2700a4cc0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe27c005e30 tx=0x7fe27c032590 comp rx=0 tx=0).stop 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 msgr2=0x7fe284197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe28000e9c0 tx=0x7fe28000ee80 comp rx=0 tx=0).stop 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 shutdown_connections 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fe2700a2800 0x7fe2700a4cc0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe284103270 0x7fe28419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2841028b0 0x7fe284197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 --2- 172.21.15.96:0/3254849337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe2841016c0 0x7fe284197930 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:16.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 >> 172.21.15.96:0/3254849337 conn(0x7fe2840fcc50 msgr2=0x7fe284105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:16.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 shutdown_connections 2024-03-18T15:48:16.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:16.695+0000 7fe28ab9d640 1 -- 172.21.15.96:0/3254849337 wait complete. 2024-03-18T15:48:16.707 INFO:tasks.cephfs.filesystem:scrub status for tag:abfc4d20-82f7-40f2-b3c9-d7337688ca3a - {'path': '/', 'tag': 'abfc4d20-82f7-40f2-b3c9-d7337688ca3a', 'options': 'recursive,force'} 2024-03-18T15:48:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:16 smithi137 ceph-mon[35868]: pgmap v598: 97 pgs: 97 active+clean; 274 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1008 KiB/s wr, 359 op/s 2024-03-18T15:48:17.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:16 smithi193 ceph-mon[36702]: pgmap v598: 97 pgs: 97 active+clean; 274 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1008 KiB/s wr, 359 op/s 2024-03-18T15:48:17.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:16 smithi096 ceph-mon[31930]: pgmap v598: 97 pgs: 97 active+clean; 274 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1008 KiB/s wr, 359 op/s 2024-03-18T15:48:18.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub queued for path: / 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: active paths [/] 2024-03-18T15:48:18.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:17 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub queued for path: / 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: active paths [/] 2024-03-18T15:48:18.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:17 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub queued for path: / 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: active paths [/] 2024-03-18T15:48:18.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:17 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:19.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:18 smithi137 ceph-mon[35868]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:19.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:18 smithi137 ceph-mon[35868]: pgmap v599: 97 pgs: 97 active+clean; 271 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 988 KiB/s wr, 303 op/s 2024-03-18T15:48:19.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:18 smithi193 ceph-mon[36702]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:19.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:18 smithi193 ceph-mon[36702]: pgmap v599: 97 pgs: 97 active+clean; 271 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 988 KiB/s wr, 303 op/s 2024-03-18T15:48:19.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:18 smithi096 ceph-mon[31930]: scrub summary: idle+waiting paths [/] 2024-03-18T15:48:19.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:18 smithi096 ceph-mon[31930]: pgmap v599: 97 pgs: 97 active+clean; 271 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 988 KiB/s wr, 303 op/s 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.357+0000 7f08af39d640 1 -- 172.21.15.96:0/2357280382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a8101720 msgr2=0x7f08a8101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.357+0000 7f08af39d640 1 --2- 172.21.15.96:0/2357280382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a8101720 0x7f08a8101ba0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0898009900 tx=0x7f089802bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- 172.21.15.96:0/2357280382 shutdown_connections 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- 172.21.15.96:0/2357280382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a81020e0 0x7f08a8108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- 172.21.15.96:0/2357280382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a8101720 0x7f08a8101ba0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- 172.21.15.96:0/2357280382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 0x7f08a8100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- 172.21.15.96:0/2357280382 >> 172.21.15.96:0/2357280382 conn(0x7f08a80fbac0 msgr2=0x7f08a80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- 172.21.15.96:0/2357280382 shutdown_connections 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- 172.21.15.96:0/2357280382 wait complete. 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 Processor -- start 2024-03-18T15:48:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- start start 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 0x7f08a80699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a8101720 0x7f08a819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810c1c0 con 0x7f08a81020e0 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810c040 con 0x7f08a8100530 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.358+0000 7f08af39d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810c340 con 0x7f08a8101720 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56280/0 (socket says 172.21.15.96:56280) 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 -- 172.21.15.96:0/2220348547 learned_addr learned my addr 172.21.15.96:0/2220348547 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08adb9a640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a8101720 0x7f08a819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08ae39b640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 0x7f08a80699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 -- 172.21.15.96:0/2220348547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a8101720 msgr2=0x7f08a819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.362 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a8101720 0x7f08a819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 -- 172.21.15.96:0/2220348547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 msgr2=0x7f08a80699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 0x7f08a80699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 -- 172.21.15.96:0/2220348547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a819f5c0 con 0x7f08a81020e0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f08a000eb10 tx=0x7f08a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f089f7fe640 1 -- 172.21.15.96:0/2220348547 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0004b20 con 0x7f08a81020e0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a819f850 con 0x7f08a81020e0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f089f7fe640 1 -- 172.21.15.96:0/2220348547 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a0004cc0 con 0x7f08a81020e0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f089f7fe640 1 -- 172.21.15.96:0/2220348547 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0010760 con 0x7f08a81020e0 2024-03-18T15:48:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.359+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a819fdc0 con 0x7f08a81020e0 2024-03-18T15:48:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.361+0000 7f089f7fe640 1 -- 172.21.15.96:0/2220348547 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f08a0005020 con 0x7f08a81020e0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 Processor -- start 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 -- start start 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819eb5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 0x560819ebadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f089f7fe640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 0x7f08940a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560819e16a40 con 0x560819eb62d0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560819dbf920 con 0x560819ebb440 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08b31e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560819e12ae0 con 0x560819eb58e0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08ae39b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 0x560819ebadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08adb9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819eb5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:44002/0 (socket says 172.21.15.96:44002) 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08ae39b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 0x560819ebadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56292/0 (socket says 172.21.15.96:56292) 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.362+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1865715992 learned_addr learned my addr 172.21.15.96:0/1865715992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08adb9a640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 0x7f08940a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f089f7fe640 1 -- 172.21.15.96:0/2220348547 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f08a00cf1d0 con 0x7f08a81020e0 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1865715992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 msgr2=0x560819eb5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819eb5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1865715992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 msgr2=0x560819ebadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 0x560819ebadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1865715992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560819dd7460 con 0x560819ebb440 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f08a00a3d50 tx=0x7f08a000efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3386216c87e0e703 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f08adb9a640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 0x7f08940a4d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f089802c7b0 tx=0x7f0898005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f089cbfe640 1 -- 172.21.15.96:0/1865715992 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a009b620 con 0x560819ebb440 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f089cbfe640 1 -- 172.21.15.96:0/1865715992 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a009e0b0 con 0x560819ebb440 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.363+0000 7f089cbfe640 1 -- 172.21.15.96:0/1865715992 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a00a1150 con 0x560819ebb440 2024-03-18T15:48:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- 172.21.15.96:0/1865715992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 msgr2=0x560819ebd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f08a00a3d50 tx=0x7f08a000efd0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- 172.21.15.96:0/1865715992 shutdown_connections 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819ebb440 0x560819ebd8d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb62d0 0x560819ebadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1865715992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819eb5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- 172.21.15.96:0/1865715992 >> 172.21.15.96:0/1865715992 conn(0x560819eaca40 msgr2=0x560819eace40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- 172.21.15.96:0/1865715992 shutdown_connections 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- 172.21.15.96:0/1865715992 wait complete. 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 -- start start 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.364+0000 7f08b31e1740 1 Processor -- start 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ebfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560819ebe3f0 con 0x560819ebb440 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560819dd2d70 con 0x560819eb62d0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560819ebe570 con 0x560819eb58e0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 10 client.0 _ll_register_callbacks cb 0x560819b2aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56294/0 (socket says 172.21.15.96:56294) 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1558974564 learned_addr learned my addr 172.21.15.96:0/1558974564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ebfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08adb9a640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 msgr2=0x560819ebfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ebfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1558974564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 msgr2=0x560819ec0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 -- 172.21.15.96:0/1558974564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560819b2b210 con 0x560819ebb440 2024-03-18T15:48:20.373 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08a00984b0 tx=0x7f08a009fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92fd492e19d663a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f0891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a00ba060 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f08a00ba200 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a009e0b0 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08a00cc050 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 10 client.17721 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560819bf7620 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.365+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a009e0b0 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f08a00b4c30 con 0x560819ebb440 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 10 client.17721 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:20.374 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 10 client.17721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 20 client.17721 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819ecb6b0 0x560819ecdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 4 client.17721 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560819e23f80 con 0x560819ecb6b0 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819ecb6b0 0x560819ecdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f0891ffb640 10 client.17721 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.366+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819ecb6b0 0x560819ecdaa0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0870eab83c0302 server_cookie=4a236ce48ef7ba32 in_seq=0 out_seq=0 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f0891ffb640 1 -- 172.21.15.96:0/1558974564 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x560819e23f80 con 0x560819ecb6b0 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f0891ffb640 10 client.17721 handle_command_reply: tid=0 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 client.17721 shutdown 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 msgr2=0x560819ec71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f08a00984b0 tx=0x7f08a009fca0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 shutdown_connections 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819ecb6b0 0x560819ecdaa0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.375 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec71c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f0891ffb640 0 client.17721 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1558974564 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ebfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 >> 172.21.15.96:0/1558974564 conn(0x560819eaca40 msgr2=0x560819eabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 shutdown_connections 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 1 -- 172.21.15.96:0/1558974564 wait complete. 2024-03-18T15:48:20.376 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.367+0000 7f08b31e1740 20 client.17721 trim_cache size 0 max 16384 2024-03-18T15:48:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.382+0000 7f08b31e1740 1 Processor -- start 2024-03-18T15:48:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.382+0000 7f08b31e1740 1 -- start start 2024-03-18T15:48:20.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.382+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.382+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.382+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819ebb440 0x560819eafca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08b31e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a6a50 con 0x560819eb58e0 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08b31e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a86b0 con 0x560819eb62d0 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08b31e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a7150 con 0x560819ebb440 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56304/0 (socket says 172.21.15.96:56304) 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 -- 172.21.15.96:0/1272569875 learned_addr learned my addr 172.21.15.96:0/1272569875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819ebb440 0x560819eafca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08adb9a640 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 -- 172.21.15.96:0/1272569875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819ebb440 msgr2=0x560819eafca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819ebb440 0x560819eafca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 -- 172.21.15.96:0/1272569875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 msgr2=0x560819ec70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 -- 172.21.15.96:0/1272569875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560819ec04d0 con 0x560819eb58e0 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f08ae39b640 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dd887576e3a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f0893fff640 1 -- 172.21.15.96:0/1272569875 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4023020 con 0x560819eb58e0 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f0893fff640 1 -- 172.21.15.96:0/1272569875 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a401f730 con 0x560819eb58e0 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.383+0000 7f0893fff640 1 -- 172.21.15.96:0/1272569875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a401d380 con 0x560819eb58e0 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- 172.21.15.96:0/1272569875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 msgr2=0x560819ebf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:20.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- 172.21.15.96:0/1272569875 shutdown_connections 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819ebb440 0x560819eafca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- 172.21.15.96:0/1272569875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819eb58e0 0x560819ebf260 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- 172.21.15.96:0/1272569875 >> 172.21.15.96:0/1272569875 conn(0x560819eaca40 msgr2=0x560819eb7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- 172.21.15.96:0/1272569875 shutdown_connections 2024-03-18T15:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- 172.21.15.96:0/1272569875 wait complete. 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- start start 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 Processor -- start 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ec4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a7fe0 con 0x560819ebb440 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a7850 con 0x560819eb62d0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f08a81a4a30 con 0x560819eb58e0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08b31e1740 10 client.0 _ll_register_callbacks cb 0x7f08a40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.384+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56314/0 (socket says 172.21.15.96:56314) 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 -- 172.21.15.96:0/115670607 learned_addr learned my addr 172.21.15.96:0/115670607 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08ae39b640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ec4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08adb9a640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 -- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 msgr2=0x560819ec4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ec4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 -- 172.21.15.96:0/115670607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 msgr2=0x560819ec5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 -- 172.21.15.96:0/115670607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560819ec6880 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08aeb9c640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f08a0098460 tx=0x7f08a009b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ceee9ca5ed6596 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a00ba040 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f08a00b4da0 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a401f730 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f08a009b440 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 10 client.17727 fetch_fsmap learned FSMap version 43 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560819f31d50 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a401f730 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8386+0+0 (secure 0 0 0) 0x7f08a00a3e80 con 0x560819ebb440 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 10 client.17727 fetch_fsmap finished waiting for FSMap version 43 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 10 client.17727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.385+0000 7f08b31e1740 20 client.17727 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:20.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819f325e0 0x560819f349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f08b31e1740 4 client.17727 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-18T15:48:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560819ebfa30 con 0x560819f325e0 2024-03-18T15:48:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f08ae39b640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819f325e0 0x560819f349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f0891ffb640 10 client.17727 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.386+0000 7f08ae39b640 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819f325e0 0x560819f349d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3813263003b135 server_cookie=25fca1d1c1d5a552 in_seq=0 out_seq=0 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f0891ffb640 1 -- 172.21.15.96:0/115670607 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560819ebfa30 con 0x560819f325e0 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f0891ffb640 10 client.17727 handle_command_reply: tid=0 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 client.17727 shutdown 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 msgr2=0x560819ec6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f08a0098460 tx=0x7f08a009b9e0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 shutdown_connections 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x560819f325e0 0x560819f349d0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560819ebb440 0x560819ec6290 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560819eb62d0 0x560819ec5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f0891ffb640 0 client.17727 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 --2- 172.21.15.96:0/115670607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560819eb58e0 0x560819ec4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.394+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 >> 172.21.15.96:0/115670607 conn(0x560819eaca40 msgr2=0x560819eb7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 shutdown_connections 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08b31e1740 1 -- 172.21.15.96:0/115670607 wait complete. 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08b31e1740 20 client.17727 trim_cache size 0 max 16384 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-03-18T15:48:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-03-18T15:48:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 msgr2=0x7f08940a4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 0x7f08940a4d30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f089802c7b0 tx=0x7f0898005fd0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.398 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 msgr2=0x7f08a819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f08a000eb10 tx=0x7f08a000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 shutdown_connections 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f08940a2870 0x7f08940a4d30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f08a81020e0 0x7f08a819ecb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08a8101720 0x7f08a819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 --2- 172.21.15.96:0/2220348547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08a8100530 0x7f08a80699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 >> 172.21.15.96:0/2220348547 conn(0x7f08a80fbac0 msgr2=0x7f08a8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 shutdown_connections 2024-03-18T15:48:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-03-18T15:48:20.395+0000 7f08af39d640 1 -- 172.21.15.96:0/2220348547 wait complete. 2024-03-18T15:48:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:20 smithi137 ceph-mon[35868]: pgmap v600: 97 pgs: 97 active+clean; 264 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-18T15:48:21.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:20 smithi193 ceph-mon[36702]: pgmap v600: 97 pgs: 97 active+clean; 264 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-18T15:48:21.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:20 smithi096 ceph-mon[31930]: pgmap v600: 97 pgs: 97 active+clean; 264 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 443 op/s 2024-03-18T15:48:22.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:22 smithi096 ceph-mon[31930]: pgmap v601: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 414 op/s 2024-03-18T15:48:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:22 smithi137 ceph-mon[35868]: pgmap v601: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 414 op/s 2024-03-18T15:48:23.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:22 smithi193 ceph-mon[36702]: pgmap v601: 97 pgs: 97 active+clean; 254 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 414 op/s 2024-03-18T15:48:23.686 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-03-18T15:48:23.687 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-18T15:48:23.960 DEBUG:teuthology.parallel:result is None 2024-03-18T15:48:23.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T15:48:23.996 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-18T15:48:23.996 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-18T15:48:24.013 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:48:24.045 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-18T15:48:24.056 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-18T15:48:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:24 smithi137 ceph-mon[35868]: pgmap v602: 97 pgs: 97 active+clean; 230 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-03-18T15:48:25.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:24 smithi193 ceph-mon[36702]: pgmap v602: 97 pgs: 97 active+clean; 230 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-03-18T15:48:25.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:24 smithi096 ceph-mon[31930]: pgmap v602: 97 pgs: 97 active+clean; 230 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-03-18T15:48:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:26 smithi137 ceph-mon[35868]: pgmap v603: 97 pgs: 97 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-03-18T15:48:27.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:26 smithi193 ceph-mon[36702]: pgmap v603: 97 pgs: 97 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-03-18T15:48:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:26 smithi096 ceph-mon[31930]: pgmap v603: 97 pgs: 97 active+clean; 215 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-03-18T15:48:28.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:27 smithi137 ceph-mon[35868]: pgmap v604: 97 pgs: 97 active+clean; 214 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-03-18T15:48:28.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:27 smithi193 ceph-mon[36702]: pgmap v604: 97 pgs: 97 active+clean; 214 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-03-18T15:48:28.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:27 smithi096 ceph-mon[31930]: pgmap v604: 97 pgs: 97 active+clean; 214 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 372 op/s 2024-03-18T15:48:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:29 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:48:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:29 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:48:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:29 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:48:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:29 smithi137 ceph-mon[35868]: scrub summary: idle 2024-03-18T15:48:30.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:29 smithi137 ceph-mon[35868]: pgmap v605: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-03-18T15:48:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:29 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:48:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:29 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:48:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:29 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:48:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:29 smithi193 ceph-mon[36702]: scrub summary: idle 2024-03-18T15:48:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:29 smithi193 ceph-mon[36702]: pgmap v605: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-03-18T15:48:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:29 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:48:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:29 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:48:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:29 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:48:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:29 smithi096 ceph-mon[31930]: scrub summary: idle 2024-03-18T15:48:30.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:29 smithi096 ceph-mon[31930]: pgmap v605: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2024-03-18T15:48:32.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:32 smithi193 ceph-mon[36702]: pgmap v606: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 555 KiB/s wr, 228 op/s 2024-03-18T15:48:32.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:32 smithi096 ceph-mon[31930]: pgmap v606: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 555 KiB/s wr, 228 op/s 2024-03-18T15:48:33.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:32 smithi137 ceph-mon[35868]: pgmap v606: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 555 KiB/s wr, 228 op/s 2024-03-18T15:48:34.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:34 smithi193 ceph-mon[36702]: pgmap v607: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 548 KiB/s wr, 156 op/s 2024-03-18T15:48:34.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:34 smithi096 ceph-mon[31930]: pgmap v607: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 548 KiB/s wr, 156 op/s 2024-03-18T15:48:35.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:34 smithi137 ceph-mon[35868]: pgmap v607: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 548 KiB/s wr, 156 op/s 2024-03-18T15:48:36.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:36 smithi193 ceph-mon[36702]: pgmap v608: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 176 KiB/s wr, 94 op/s 2024-03-18T15:48:36.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:36 smithi096 ceph-mon[31930]: pgmap v608: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 176 KiB/s wr, 94 op/s 2024-03-18T15:48:37.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:36 smithi137 ceph-mon[35868]: pgmap v608: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 176 KiB/s wr, 94 op/s 2024-03-18T15:48:37.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:37 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:48:38.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:37 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:48:38.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:37 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-18T15:48:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:38 smithi096 ceph-mon[31930]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 KiB/s wr, 38 op/s 2024-03-18T15:48:38.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:48:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:48:38.937 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:38 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:48:39.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:38 smithi137 ceph-mon[35868]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 KiB/s wr, 38 op/s 2024-03-18T15:48:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:48:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:48:39.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:38 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:48:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:38 smithi193 ceph-mon[36702]: pgmap v609: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 13 KiB/s wr, 38 op/s 2024-03-18T15:48:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-18T15:48:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-18T15:48:39.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:38 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' 2024-03-18T15:48:40.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:40 smithi096 ceph-mon[31930]: pgmap v610: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 KiB/s wr, 34 op/s 2024-03-18T15:48:41.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:40 smithi137 ceph-mon[35868]: pgmap v610: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 KiB/s wr, 34 op/s 2024-03-18T15:48:41.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:40 smithi193 ceph-mon[36702]: pgmap v610: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 13 KiB/s wr, 34 op/s 2024-03-18T15:48:42.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:42 smithi096 ceph-mon[31930]: pgmap v611: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:43.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:42 smithi137 ceph-mon[35868]: pgmap v611: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:43.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:42 smithi193 ceph-mon[36702]: pgmap v611: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:44.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:44 smithi096 ceph-mon[31930]: pgmap v612: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:45.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:44 smithi137 ceph-mon[35868]: pgmap v612: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:45.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:44 smithi193 ceph-mon[36702]: pgmap v612: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-03-18T15:48:46.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-18T15:48:46.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 -- 172.21.15.96:0/3929865539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401017f0 msgr2=0x7f0d40101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 --2- 172.21.15.96:0/3929865539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401017f0 0x7f0d40101c10 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d3002bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 -- 172.21.15.96:0/3929865539 shutdown_connections 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 --2- 172.21.15.96:0/3929865539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401033a0 0x7f0d40109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 --2- 172.21.15.96:0/3929865539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401029e0 0x7f0d40102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 --2- 172.21.15.96:0/3929865539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401017f0 0x7f0d40101c10 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 -- 172.21.15.96:0/3929865539 >> 172.21.15.96:0/3929865539 conn(0x7f0d400fcd60 msgr2=0x7f0d400ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 -- 172.21.15.96:0/3929865539 shutdown_connections 2024-03-18T15:48:46.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.823+0000 7f0d4533b640 1 -- 172.21.15.96:0/3929865539 wait complete. 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 Processor -- start 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 -- start start 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401017f0 0x7f0d40197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401029e0 0x7f0d40198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d470 con 0x7f0d401033a0 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d2f0 con 0x7f0d401017f0 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d4533b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d5f0 con 0x7f0d401029e0 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40990/0 (socket says 172.21.15.96:40990) 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d3f7fe640 1 -- 172.21.15.96:0/1004151282 learned_addr learned my addr 172.21.15.96:0/1004151282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d3effd640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401017f0 0x7f0d40197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.824+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401029e0 0x7f0d40198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 -- 172.21.15.96:0/1004151282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401029e0 msgr2=0x7f0d40198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401029e0 0x7f0d40198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 -- 172.21.15.96:0/1004151282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401017f0 msgr2=0x7f0d40197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401017f0 0x7f0d40197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 -- 172.21.15.96:0/1004151282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4019f760 con 0x7f0d401033a0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800eb10 tx=0x7f0d3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d27fff640 1 -- 172.21.15.96:0/1004151282 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38004b20 con 0x7f0d401033a0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d27fff640 1 -- 172.21.15.96:0/1004151282 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d38004cc0 con 0x7f0d401033a0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4019fa50 con 0x7f0d401033a0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d27fff640 1 -- 172.21.15.96:0/1004151282 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38010760 con 0x7f0d401033a0 2024-03-18T15:48:46.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.825+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4019ffa0 con 0x7f0d401033a0 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.827+0000 7f0d27fff640 1 -- 172.21.15.96:0/1004151282 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0d38005020 con 0x7f0d401033a0 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 Processor -- start 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 -- start start 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d27fff640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 0x7f0d280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7ace6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7ace9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f7ac46500 con 0x562f7ace1b60 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f7abeebe0 con 0x562f7ace6cd0 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d4800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f7ac425a0 con 0x562f7ace1170 2024-03-18T15:48:46.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 0x7f0d280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d27fff640 1 -- 172.21.15.96:0/1004151282 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0d380cf170 con 0x7f0d401033a0 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42398/0 (socket says 172.21.15.96:42398) 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 -- 172.21.15.96:0/4249956939 learned_addr learned my addr 172.21.15.96:0/4249956939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7ace9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7ace6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 -- 172.21.15.96:0/4249956939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 msgr2=0x562f7ace9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7ace9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 -- 172.21.15.96:0/4249956939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 msgr2=0x562f7ace6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7ace6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.828+0000 7f0d3effd640 1 -- 172.21.15.96:0/4249956939 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7ac024b0 con 0x562f7ace1170 2024-03-18T15:48:46.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 0x7f0d280a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d40199050 tx=0x7f0d34009b70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d30009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb756a6dc56ae1de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d257fa640 1 -- 172.21.15.96:0/4249956939 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3002cb80 con 0x562f7ace1170 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d257fa640 1 -- 172.21.15.96:0/4249956939 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30005040 con 0x562f7ace1170 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d257fa640 1 -- 172.21.15.96:0/4249956939 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300048c0 con 0x562f7ace1170 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d4800a740 1 -- 172.21.15.96:0/4249956939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 msgr2=0x562f7ace1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.829+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d30009560 comp rx=0 tx=0).stop 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- 172.21.15.96:0/4249956939 shutdown_connections 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7ace9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7ace6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4249956939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1170 0x562f7ace1590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- 172.21.15.96:0/4249956939 >> 172.21.15.96:0/4249956939 conn(0x562f7acdc6e0 msgr2=0x562f7acdcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- 172.21.15.96:0/4249956939 shutdown_connections 2024-03-18T15:48:46.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- 172.21.15.96:0/4249956939 wait complete. 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- start start 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 Processor -- start 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 0x562f7acebba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7acf2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f7ace9c20 con 0x562f7ace1170 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562f7ac02160 con 0x562f7ace6cd0 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562f7ace9da0 con 0x562f7ace1b60 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d4800a740 10 client.0 _ll_register_callbacks cb 0x562f7abe6150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41010/0 (socket says 172.21.15.96:41010) 2024-03-18T15:48:46.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d3e7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 0x562f7acebba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d3effd640 1 -- 172.21.15.96:0/4120752515 learned_addr learned my addr 172.21.15.96:0/4120752515 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.830+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7acf2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 -- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 msgr2=0x562f7acebba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 0x562f7acebba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 -- 172.21.15.96:0/4120752515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 msgr2=0x562f7acf2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7acf2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 -- 172.21.15.96:0/4120752515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7aa247d0 con 0x562f7ace1170 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 0x562f7acebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d300092a0 tx=0x7f0d30005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044054d02624ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:46.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300055f0 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0d30004240 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30005040 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0d30055050 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d4800a740 10 client.17739 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562f7a95a3f0 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d30005040 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.831+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f0d300038f0 con 0x562f7ace1170 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 10 client.17739 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 20 client.17739 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7acf6d40 0x562f7acf9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 4 client.17739 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f7ac53a40 con 0x562f7acf6d40 2024-03-18T15:48:46.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7acf6d40 0x562f7acf9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d22ff5640 10 client.17739 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.832+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7acf6d40 0x562f7acf9130 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523f9dc0541e761c server_cookie=ea0c1540d661e85a in_seq=0 out_seq=0 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4120752515 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x562f7ac53a40 con 0x562f7acf6d40 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d22ff5640 10 client.17739 handle_command_reply: tid=0 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 client.17739 shutdown 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 msgr2=0x562f7aceb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d300092a0 tx=0x7f0d30005480 comp rx=0 tx=0).stop 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 shutdown_connections 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace6cd0 0x562f7acf2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace1b60 0x562f7acebba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7acf6d40 0x562f7acf9130 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4120752515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7aceb660 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d22ff5640 0 client.17739 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 >> 172.21.15.96:0/4120752515 conn(0x562f7acdc6e0 msgr2=0x562f7acdb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 shutdown_connections 2024-03-18T15:48:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.833+0000 7f0d4800a740 1 -- 172.21.15.96:0/4120752515 wait complete. 2024-03-18T15:48:46.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.834+0000 7f0d4800a740 20 client.17739 trim_cache size 0 max 16384 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 Processor -- start 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 -- start start 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1170 0x562f7aceac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7aceb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a8df0 con 0x562f7ace1b60 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401aaa50 con 0x562f7ace1170 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d4800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a94f0 con 0x562f7ace6cd0 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1170 0x562f7aceac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41026/0 (socket says 172.21.15.96:41026) 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 -- 172.21.15.96:0/1632403690 learned_addr learned my addr 172.21.15.96:0/1632403690 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7aceb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 -- 172.21.15.96:0/1632403690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 msgr2=0x562f7aceb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7aceb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 -- 172.21.15.96:0/1632403690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1170 msgr2=0x562f7aceac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1170 0x562f7aceac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 -- 172.21.15.96:0/1632403690 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f7a91a0f0 con 0x562f7ace1b60 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0d340095f0 tx=0x7f0d34021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389c33054b434d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d24ff9640 1 -- 172.21.15.96:0/1632403690 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3402c450 con 0x562f7ace1b60 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d24ff9640 1 -- 172.21.15.96:0/1632403690 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3402c5f0 con 0x562f7ace1b60 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.851+0000 7f0d24ff9640 1 -- 172.21.15.96:0/1632403690 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340038a0 con 0x562f7ace1b60 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- 172.21.15.96:0/1632403690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 msgr2=0x562f7acf2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0d340095f0 tx=0x7f0d34021000 comp rx=0 tx=0).stop 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- 172.21.15.96:0/1632403690 shutdown_connections 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7aceb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1b60 0x562f7acf2ba0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 --2- 172.21.15.96:0/1632403690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1170 0x562f7aceac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- 172.21.15.96:0/1632403690 >> 172.21.15.96:0/1632403690 conn(0x562f7acdc6e0 msgr2=0x562f7ace01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- 172.21.15.96:0/1632403690 shutdown_connections 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- 172.21.15.96:0/1632403690 wait complete. 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 -- start start 2024-03-18T15:48:46.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.852+0000 7f0d4800a740 1 Processor -- start 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 0x562f7acee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7acf1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401aa380 con 0x562f7ace1170 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a9bf0 con 0x562f7ace1b60 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401a6dd0 con 0x562f7ace6cd0 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 10 client.0 _ll_register_callbacks cb 0x7f0d340114c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41030/0 (socket says 172.21.15.96:41030) 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 -- 172.21.15.96:0/4011489090 learned_addr learned my addr 172.21.15.96:0/4011489090 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3f7fe640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7acf1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 0x562f7acee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 -- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 msgr2=0x562f7acf1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7acf1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 -- 172.21.15.96:0/4011489090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 msgr2=0x562f7acee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 0x562f7acee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 -- 172.21.15.96:0/4011489090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f7ac0d9d0 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 0x562f7acee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d3effd640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0d3002c8e0 tx=0x7f0d3002c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6049757f6c5d9eb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30000e60 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0d3002cc90 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3402c5f0 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0d30004240 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 10 client.17745 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.853+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562f7acf2ad0 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d3402c5f0 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f0d30061030 con 0x562f7ace1170 2024-03-18T15:48:46.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 10 client.17745 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 20 client.17745 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7ad618f0 0x562f7ad63d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 4 client.17745 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f7ac4dd90 con 0x562f7ad618f0 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7ad618f0 0x562f7ad63d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d22ff5640 10 client.17745 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d3e7fc640 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7ad618f0 0x562f7ad63d10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b8089f8ba0abb server_cookie=686bf99beff2ae73 in_seq=0 out_seq=0 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d22ff5640 1 -- 172.21.15.96:0/4011489090 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562f7ac4dd90 con 0x562f7ad618f0 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d22ff5640 10 client.17745 handle_command_reply: tid=0 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.854+0000 7f0d4800a740 1 client.17745 shutdown 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 msgr2=0x562f7acede10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0d3002c8e0 tx=0x7f0d3002c540 comp rx=0 tx=0).stop 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 shutdown_connections 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562f7ace6cd0 0x562f7acf1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f7ace1b60 0x562f7acee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x562f7ad618f0 0x562f7ad63d10 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 --2- 172.21.15.96:0/4011489090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f7ace1170 0x562f7acede10 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d22ff5640 0 client.17745 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 >> 172.21.15.96:0/4011489090 conn(0x562f7acdc6e0 msgr2=0x562f7acdb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 shutdown_connections 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 1 -- 172.21.15.96:0/4011489090 wait complete. 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4800a740 20 client.17745 trim_cache size 0 max 16384 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:46.864 INFO:teuthology.orchestra.run.smithi096.stdout: "status": "no active scrubs running", 2024-03-18T15:48:46.865 INFO:teuthology.orchestra.run.smithi096.stdout: "scrubs": {} 2024-03-18T15:48:46.865 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 msgr2=0x7f0d280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 0x7f0d280a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d40199050 tx=0x7f0d34009b70 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 msgr2=0x7f0d4019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800eb10 tx=0x7f0d3800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 shutdown_connections 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0d280a2800 0x7f0d280a4cc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d401033a0 0x7f0d4019f0f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d401029e0 0x7f0d40198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 --2- 172.21.15.96:0/1004151282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0d401017f0 0x7f0d40197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 >> 172.21.15.96:0/1004151282 conn(0x7f0d400fcd60 msgr2=0x7f0d40105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 shutdown_connections 2024-03-18T15:48:46.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.855+0000 7f0d4533b640 1 -- 172.21.15.96:0/1004151282 wait complete. 2024-03-18T15:48:46.867 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-18T15:48:46.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:48:46.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:46 smithi096 ceph-mon[31930]: pgmap v613: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:46.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- 172.21.15.96:0/1004942447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c101800 msgr2=0x7f593c101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- 172.21.15.96:0/1004942447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c101800 0x7f593c101c20 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f592c009900 tx=0x7f592c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- 172.21.15.96:0/1004942447 shutdown_connections 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- 172.21.15.96:0/1004942447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c1033b0 0x7f593c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- 172.21.15.96:0/1004942447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 0x7f593c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- 172.21.15.96:0/1004942447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c101800 0x7f593c101c20 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- 172.21.15.96:0/1004942447 >> 172.21.15.96:0/1004942447 conn(0x7f593c0fcd90 msgr2=0x7f593c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- 172.21.15.96:0/1004942447 shutdown_connections 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- 172.21.15.96:0/1004942447 wait complete. 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 Processor -- start 2024-03-18T15:48:46.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- start start 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c101800 0x7f593c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 0x7f593c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d4a0 con 0x7f593c1033b0 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d320 con 0x7f593c101800 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.969+0000 7f5944675640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d620 con 0x7f593c1029f0 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41058/0 (socket says 172.21.15.96:41058) 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 -- 172.21.15.96:0/3644912792 learned_addr learned my addr 172.21.15.96:0/3644912792 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942411640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c101800 0x7f593c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5941c10640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 0x7f593c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 -- 172.21.15.96:0/3644912792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 msgr2=0x7f593c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 0x7f593c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 -- 172.21.15.96:0/3644912792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c101800 msgr2=0x7f593c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c101800 0x7f593c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 -- 172.21.15.96:0/3644912792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c19f740 con 0x7f593c1033b0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5942c12640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f593400efd0 tx=0x7f593400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934003b70 con 0x7f593c1033b0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934003d10 con 0x7f593c1033b0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c19fa30 con 0x7f593c1033b0 2024-03-18T15:48:46.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59340109f0 con 0x7f593c1033b0 2024-03-18T15:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.970+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c19ff30 con 0x7f593c1033b0 2024-03-18T15:48:46.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.971+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c108ba0 con 0x7f593c1033b0 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.972+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5934010b90 con 0x7f593c1033b0 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.973+0000 7f59337fe640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 0x7f59280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.973+0000 7f5942411640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 0x7f59280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.973+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f59340cf290 con 0x7f593c1033b0 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.973+0000 7f5942411640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 0x7f59280a4cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f592c02c170 tx=0x7f592c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:46.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:46.976+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f593409c300 con 0x7f593c1033b0 2024-03-18T15:48:47.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:46 smithi137 ceph-mon[35868]: pgmap v613: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:47.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:46 smithi193 ceph-mon[36702]: pgmap v613: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.191+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f593c02d400 con 0x7f593c1033b0 2024-03-18T15:48:47.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.191+0000 7f59337fe640 1 -- 172.21.15.96:0/3644912792 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11952 (secure 0 0 0) 0x7f593409bc00 con 0x7f593c1033b0 2024-03-18T15:48:47.193 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:48:47.193 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:48:28.755036+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24460,24452,24485,24472,24493]},"id":1}]} 2024-03-18T15:48:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 44 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 msgr2=0x7f59280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 0x7f59280a4cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f592c02c170 tx=0x7f592c02df90 comp rx=0 tx=0).stop 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 msgr2=0x7f593c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f593400efd0 tx=0x7f593400c6a0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 shutdown_connections 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f59280a2800 0x7f59280a4cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593c1033b0 0x7f593c19f0d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f593c1029f0 0x7f593c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 --2- 172.21.15.96:0/3644912792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f593c101800 0x7f593c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 >> 172.21.15.96:0/3644912792 conn(0x7f593c0fcd90 msgr2=0x7f593c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 shutdown_connections 2024-03-18T15:48:47.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.194+0000 7f5944675640 1 -- 172.21.15.96:0/3644912792 wait complete. 2024-03-18T15:48:47.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-03-18T15:48:47.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- 172.21.15.96:0/3115256458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd0100700 msgr2=0x7efcd0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- 172.21.15.96:0/3115256458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd0100700 0x7efcd0100b20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- 172.21.15.96:0/3115256458 shutdown_connections 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- 172.21.15.96:0/3115256458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01022b0 0x7efcd0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- 172.21.15.96:0/3115256458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd01018f0 0x7efcd0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- 172.21.15.96:0/3115256458 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd0100700 0x7efcd0100b20 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- 172.21.15.96:0/3115256458 >> 172.21.15.96:0/3115256458 conn(0x7efcd00fbc90 msgr2=0x7efcd00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- 172.21.15.96:0/3115256458 shutdown_connections 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- 172.21.15.96:0/3115256458 wait complete. 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 Processor -- start 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- start start 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd0100700 0x7efcd0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01018f0 0x7efcd0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d460 con 0x7efcd01022b0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d2e0 con 0x7efcd01018f0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.309+0000 7efcd82a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010d5e0 con 0x7efcd0100700 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41076/0 (socket says 172.21.15.96:41076) 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 -- 172.21.15.96:0/190639622 learned_addr learned my addr 172.21.15.96:0/190639622 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd603d640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd0100700 0x7efcd0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd583c640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01018f0 0x7efcd0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 -- 172.21.15.96:0/190639622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd0100700 msgr2=0x7efcd0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd0100700 0x7efcd0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 -- 172.21.15.96:0/190639622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01018f0 msgr2=0x7efcd0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01018f0 0x7efcd0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 -- 172.21.15.96:0/190639622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019f760 con 0x7efcd01022b0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd683e640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7efcc800eb10 tx=0x7efcc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcc77fe640 1 -- 172.21.15.96:0/190639622 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8004b20 con 0x7efcd01022b0 2024-03-18T15:48:47.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019fa50 con 0x7efcd01022b0 2024-03-18T15:48:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcc77fe640 1 -- 172.21.15.96:0/190639622 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc8004cc0 con 0x7efcd01022b0 2024-03-18T15:48:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcc77fe640 1 -- 172.21.15.96:0/190639622 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8010760 con 0x7efcd01022b0 2024-03-18T15:48:47.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.310+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd019ff50 con 0x7efcd01022b0 2024-03-18T15:48:47.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.312+0000 7efcc77fe640 1 -- 172.21.15.96:0/190639622 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7efcc8005020 con 0x7efcd01022b0 2024-03-18T15:48:47.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 Processor -- start 2024-03-18T15:48:47.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 -- start start 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bab0335490 con 0x55bab03da7c0 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bab02ddc90 con 0x55bab03d5650 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bab0331530 con 0x55bab03d4c60 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcc77fe640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 0x7efcbc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcd603d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcd583c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcd603d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:43974/0 (socket says 172.21.15.96:43974) 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.313+0000 7efcd603d640 1 -- 172.21.15.96:0/238948678 learned_addr learned my addr 172.21.15.96:0/238948678 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd583c640 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd683e640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 0x7efcbc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 -- 172.21.15.96:0/238948678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 msgr2=0x55bab03d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03d5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 -- 172.21.15.96:0/238948678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 msgr2=0x55bab03dcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03dcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 -- 172.21.15.96:0/238948678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab0398500 con 0x55bab03d5650 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcc77fe640 1 -- 172.21.15.96:0/190639622 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efcc80cf000 con 0x7efcd01022b0 2024-03-18T15:48:47.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd603d640 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc0095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b6af29a473a300 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcd683e640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 0x7efcbc0a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efcc8097dd0 tx=0x7efcc80c63c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcc4bfe640 1 -- 172.21.15.96:0/238948678 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc02cb80 con 0x55bab03d5650 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcc4bfe640 1 -- 172.21.15.96:0/238948678 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc005040 con 0x55bab03d5650 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.314+0000 7efcc4bfe640 1 -- 172.21.15.96:0/238948678 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc004920 con 0x55bab03d5650 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- 172.21.15.96:0/238948678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 msgr2=0x55bab03da150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc0095e0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- 172.21.15.96:0/238948678 shutdown_connections 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03dcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03da150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/238948678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- 172.21.15.96:0/238948678 >> 172.21.15.96:0/238948678 conn(0x55bab03cbdc0 msgr2=0x55bab03cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- 172.21.15.96:0/238948678 shutdown_connections 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- 172.21.15.96:0/238948678 wait complete. 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 -- start start 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.315+0000 7efcdaf6a740 1 Processor -- start 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bab03dd710 con 0x55bab03da7c0 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bab02f1230 con 0x55bab03d5650 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bab03dd890 con 0x55bab03d4c60 2024-03-18T15:48:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 10 client.0 _ll_register_callbacks cb 0x55bab032d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41102/0 (socket says 172.21.15.96:41102) 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 -- 172.21.15.96:0/2780208179 learned_addr learned my addr 172.21.15.96:0/2780208179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd603d640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 -- 172.21.15.96:0/2780208179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 msgr2=0x55bab03df150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03df150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 -- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 msgr2=0x55bab03df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03df690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 -- 172.21.15.96:0/2780208179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab0300100 con 0x55bab03da7c0 2024-03-18T15:48:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcd583c640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7efcc0012ae0 tx=0x7efcc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edde7e029ccf7a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc000eae0 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efcc000ec80 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efccc005040 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.316+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efcc000ee80 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 10 client.17763 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bab00493f0 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc005040 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7efcc002b070 con 0x55bab03da7c0 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 10 client.17763 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 10 client.17763 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:48:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 20 client.17763 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab03eaf90 0x55bab03ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 4 client.17763 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bab03429d0 con 0x55bab03eaf90 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcd603d640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab03eaf90 0x55bab03ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcb9ffb640 10 client.17763 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.317+0000 7efcd603d640 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab03eaf90 0x55bab03ed430 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c1822e198d530ce server_cookie=2679cb51f2406678 in_seq=0 out_seq=0 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcb9ffb640 1 -- 172.21.15.96:0/2780208179 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x7efccc0509c0 con 0x55bab03eaf90 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcb9ffb640 10 client.17763 handle_command_reply: tid=0 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 client.17763 shutdown 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 msgr2=0x55bab03e66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7efcc0012ae0 tx=0x7efcc0004730 comp rx=0 tx=0).stop 2024-03-18T15:48:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 shutdown_connections 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab03eaf90 0x55bab03ed430 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03da7c0 0x55bab03e66d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcb9ffb640 0 client.17763 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/2780208179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03df150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 >> 172.21.15.96:0/2780208179 conn(0x55bab03cbdc0 msgr2=0x55bab03cafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 shutdown_connections 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 1 -- 172.21.15.96:0/2780208179 wait complete. 2024-03-18T15:48:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.318+0000 7efcdaf6a740 20 client.17763 trim_cache size 0 max 16384 2024-03-18T15:48:47.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.331+0000 7efcdaf6a740 1 Processor -- start 2024-03-18T15:48:47.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.331+0000 7efcdaf6a740 1 -- start start 2024-03-18T15:48:47.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03da7c0 0x55bab03f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a6bd0 con 0x55bab03d5650 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a8830 con 0x55bab03da7c0 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a72d0 con 0x55bab03d4c60 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41112/0 (socket says 172.21.15.96:41112) 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 -- 172.21.15.96:0/1707888905 learned_addr learned my addr 172.21.15.96:0/1707888905 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd583c640 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd583c640 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03da7c0 0x55bab03f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 -- 172.21.15.96:0/1707888905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 msgr2=0x55bab03cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 -- 172.21.15.96:0/1707888905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03da7c0 msgr2=0x55bab03f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03da7c0 0x55bab03f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 -- 172.21.15.96:0/1707888905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab01137d0 con 0x55bab03d5650 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcd603d640 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efccc02c710 tx=0x7efccc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08245432f1c109c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcbbfff640 1 -- 172.21.15.96:0/1707888905 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc054070 con 0x55bab03d5650 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcbbfff640 1 -- 172.21.15.96:0/1707888905 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc04b040 con 0x55bab03d5650 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.332+0000 7efcbbfff640 1 -- 172.21.15.96:0/1707888905 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc005540 con 0x55bab03d5650 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- 172.21.15.96:0/1707888905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 msgr2=0x55bab03e6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efccc02c710 tx=0x7efccc000bc0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- 172.21.15.96:0/1707888905 shutdown_connections 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03da7c0 0x55bab03f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d5650 0x55bab03e6690 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/1707888905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03d4c60 0x55bab03cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- 172.21.15.96:0/1707888905 >> 172.21.15.96:0/1707888905 conn(0x55bab03cbdc0 msgr2=0x55bab03ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- 172.21.15.96:0/1707888905 shutdown_connections 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- 172.21.15.96:0/1707888905 wait complete. 2024-03-18T15:48:47.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- start start 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 Processor -- start 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03da7c0 0x55bab03e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a8160 con 0x55bab03d4c60 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a79d0 con 0x55bab03d5650 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01a4bb0 con 0x55bab03da7c0 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.333+0000 7efcdaf6a740 10 client.0 _ll_register_callbacks cb 0x7efcc000bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41114/0 (socket says 172.21.15.96:41114) 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 -- 172.21.15.96:0/148244080 learned_addr learned my addr 172.21.15.96:0/148244080 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd603d640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03da7c0 0x55bab03e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 -- 172.21.15.96:0/148244080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03da7c0 msgr2=0x55bab03e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03da7c0 0x55bab03e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 -- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 msgr2=0x55bab03e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 -- 172.21.15.96:0/148244080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab03e5da0 con 0x55bab03d4c60 2024-03-18T15:48:47.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcd583c640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efcc000bdd0 tx=0x7efcc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d75149d2d82090 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0012860 con 0x55bab03d4c60 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7efcc0036040 con 0x55bab03d4c60 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efccc04b040 con 0x55bab03d4c60 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7efcc0040050 con 0x55bab03d4c60 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcdaf6a740 10 client.17769 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55bab04517b0 con 0x55bab03d4c60 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc04b040 con 0x55bab03d4c60 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.334+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7efcc002b040 con 0x55bab03d4c60 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 10 client.17769 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 10 client.17769 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 20 client.17769 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab0452210 0x55bab0454630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 4 client.17769 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 --> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bab033cd20 con 0x55bab0452210 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcd603d640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab0452210 0x55bab0454630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcb9ffb640 10 client.17769 ms_handle_connect on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcd603d640 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab0452210 0x55bab0454630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7bb3b944b9fb49d server_cookie=e5943aec370389d8 in_seq=0 out_seq=0 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcb9ffb640 1 -- 172.21.15.96:0/148244080 <== mds.4 v2:172.21.15.137:6836/2776916439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bab033cd20 con 0x55bab0452210 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcb9ffb640 10 client.17769 handle_command_reply: tid=0 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 client.17769 shutdown 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 msgr2=0x55bab03e4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efcc000bdd0 tx=0x7efcc0004730 comp rx=0 tx=0).stop 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 shutdown_connections 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:6836/2776916439,v1:172.21.15.137:6837/2776916439] conn(0x55bab0452210 0x55bab0454630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bab03da7c0 0x55bab03e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bab03d5650 0x55bab03e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 --2- 172.21.15.96:0/148244080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bab03d4c60 0x55bab03e4260 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcb9ffb640 0 client.17769 ms_handle_reset on v2:172.21.15.137:6836/2776916439 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 >> 172.21.15.96:0/148244080 conn(0x55bab03cbdc0 msgr2=0x55bab03cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 shutdown_connections 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.335+0000 7efcdaf6a740 1 -- 172.21.15.96:0/148244080 wait complete. 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcdaf6a740 20 client.17769 trim_cache size 0 max 16384 2024-03-18T15:48:47.347 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 msgr2=0x7efcbc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 0x7efcbc0a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efcc8097dd0 tx=0x7efcc80c63c0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 msgr2=0x7efcd019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7efcc800eb10 tx=0x7efcc800c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 shutdown_connections 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7efcbc0a2800 0x7efcbc0a4cc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcd01022b0 0x7efcd019f0f0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efcd01018f0 0x7efcd0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 --2- 172.21.15.96:0/190639622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcd0100700 0x7efcd0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 >> 172.21.15.96:0/190639622 conn(0x7efcd00fbc90 msgr2=0x7efcd0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 shutdown_connections 2024-03-18T15:48:47.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.336+0000 7efcd82a1640 1 -- 172.21.15.96:0/190639622 wait complete. 2024-03-18T15:48:47.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-03-18T15:48:47.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 -- 172.21.15.96:0/2097501191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5881018d0 msgr2=0x7fd588101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 --2- 172.21.15.96:0/2097501191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5881018d0 0x7fd588101d50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd578009900 tx=0x7fd57802bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:47.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 -- 172.21.15.96:0/2097501191 shutdown_connections 2024-03-18T15:48:47.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 --2- 172.21.15.96:0/2097501191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd588102290 0x7fd588108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 --2- 172.21.15.96:0/2097501191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd5881018d0 0x7fd588101d50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 --2- 172.21.15.96:0/2097501191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 -- 172.21.15.96:0/2097501191 >> 172.21.15.96:0/2097501191 conn(0x7fd5880fbc70 msgr2=0x7fd5880fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 -- 172.21.15.96:0/2097501191 shutdown_connections 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.453+0000 7fd58f819640 1 -- 172.21.15.96:0/2097501191 wait complete. 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 Processor -- start 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 -- start start 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5881018d0 0x7fd588196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810c360 con 0x7fd588102290 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810c1e0 con 0x7fd5881006e0 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58f819640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd58810c4e0 con 0x7fd5881018d0 2024-03-18T15:48:47.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41134/0 (socket says 172.21.15.96:41134) 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 -- 172.21.15.96:0/2068466915 learned_addr learned my addr 172.21.15.96:0/2068466915 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.96:44014/0 (socket says 172.21.15.96:44014) 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5881018d0 0x7fd588196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 -- 172.21.15.96:0/2068466915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5881018d0 msgr2=0x7fd588196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5881018d0 0x7fd588196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 -- 172.21.15.96:0/2068466915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 msgr2=0x7fd588196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 -- 172.21.15.96:0/2068466915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58819e620 con 0x7fd588102290 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd58000eb10 tx=0x7fd58000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.454+0000 7fd57e7fc640 1 -- 172.21.15.96:0/2068466915 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580004b20 con 0x7fd588102290 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.455+0000 7fd57e7fc640 1 -- 172.21.15.96:0/2068466915 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd580004cc0 con 0x7fd588102290 2024-03-18T15:48:47.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.455+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58819e910 con 0x7fd588102290 2024-03-18T15:48:47.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.455+0000 7fd57e7fc640 1 -- 172.21.15.96:0/2068466915 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580010760 con 0x7fd588102290 2024-03-18T15:48:47.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.455+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58819ee30 con 0x7fd588102290 2024-03-18T15:48:47.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.457+0000 7fd57e7fc640 1 -- 172.21.15.96:0/2068466915 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fd580005020 con 0x7fd588102290 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.457+0000 7fd592408740 1 Processor -- start 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.457+0000 7fd592408740 1 -- start start 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ec3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd57e7fc640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ec8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 0x557b21ecb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b21e28490 con 0x557b21ec3850 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b21dd0c90 con 0x557b21ec4240 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd592408740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557b21e24530 con 0x557b21ec93b0 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58cdb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ec3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58cdb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ec3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41144/0 (socket says 172.21.15.96:41144) 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58cdb4640 1 -- 172.21.15.96:0/1803203393 learned_addr learned my addr 172.21.15.96:0/1803203393 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd57e7fc640 1 -- 172.21.15.96:0/2068466915 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd5800cf170 con 0x7fd588102290 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 0x557b21ecb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ec8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 -- 172.21.15.96:0/1803203393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 msgr2=0x557b21ec8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ec8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 -- 172.21.15.96:0/1803203393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 msgr2=0x557b21ec3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ec3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 -- 172.21.15.96:0/1803203393 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b21e8b500 con 0x557b21ec93b0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 0x7fd5740a4cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd588101730 tx=0x7fd584005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 0x557b21ecb840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5800ccc90 tx=0x7fd580004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b6338e730cd472a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd5737fe640 1 -- 172.21.15.96:0/1803203393 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580075070 con 0x557b21ec93b0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd5737fe640 1 -- 172.21.15.96:0/1803203393 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58009c1e0 con 0x557b21ec93b0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.458+0000 7fd5737fe640 1 -- 172.21.15.96:0/1803203393 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580098150 con 0x557b21ec93b0 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- 172.21.15.96:0/1803203393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 msgr2=0x557b21ecb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 0x557b21ecb840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd5800ccc90 tx=0x7fd580004f20 comp rx=0 tx=0).stop 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- 172.21.15.96:0/1803203393 shutdown_connections 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec93b0 0x557b21ecb840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ec8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 --2- 172.21.15.96:0/1803203393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ec3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- 172.21.15.96:0/1803203393 >> 172.21.15.96:0/1803203393 conn(0x557b21ebedc0 msgr2=0x557b21ebf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- 172.21.15.96:0/1803203393 shutdown_connections 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- 172.21.15.96:0/1803203393 wait complete. 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 -- start start 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.459+0000 7fd592408740 1 Processor -- start 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec4240 0x557b21ece1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ed5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557b21ecc360 con 0x557b21ec3850 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b21de4230 con 0x557b21ec93b0 2024-03-18T15:48:47.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557b21ecc4e0 con 0x557b21ec4240 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd592408740 10 client.0 _ll_register_callbacks cb 0x557b21e20010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41146/0 (socket says 172.21.15.96:41146) 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 -- 172.21.15.96:0/3090837960 learned_addr learned my addr 172.21.15.96:0/3090837960 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec4240 0x557b21ece1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ed5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 -- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec4240 msgr2=0x557b21ece1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec4240 0x557b21ece1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 -- 172.21.15.96:0/3090837960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 msgr2=0x557b21ed5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ed5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 -- 172.21.15.96:0/3090837960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b21df3100 con 0x557b21ec3850 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd584003680 tx=0x7fd584004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4adf39e3df6546f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.460+0000 7fd570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58400e6f0 con 0x557b21ec3850 2024-03-18T15:48:47.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd58400e890 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58009c1e0 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd58400ea90 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 10 client.17781 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b21b3c3f0 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58009c1e0 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7fd58401a070 con 0x557b21ec3850 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 10 client.17781 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 10 client.17781 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 20 client.17781 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21edbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 4 client.17781 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b21e359d0 con 0x557b21ed9b40 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.461+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21edbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.462+0000 7fd570ff9640 10 client.17781 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.462+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21edbfe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4982dc47368d5386 server_cookie=5bcb9377b5506d72 in_seq=0 out_seq=0 2024-03-18T15:48:47.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd570ff9640 1 -- 172.21.15.96:0/3090837960 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x557b21e359d0 con 0x557b21ed9b40 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd570ff9640 10 client.17781 handle_command_reply: tid=0 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 client.17781 shutdown 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 msgr2=0x557b21ecdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd584003680 tx=0x7fd584004730 comp rx=0 tx=0).stop 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 shutdown_connections 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21edbfe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ed5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec4240 0x557b21ece1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd570ff9640 0 client.17781 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 --2- 172.21.15.96:0/3090837960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec3850 0x557b21ecdca0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 >> 172.21.15.96:0/3090837960 conn(0x557b21ebedc0 msgr2=0x557b21ebdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 shutdown_connections 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 1 -- 172.21.15.96:0/3090837960 wait complete. 2024-03-18T15:48:47.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.463+0000 7fd592408740 20 client.17781 trim_cache size 0 max 16384 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 Processor -- start 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 -- start start 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ebf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ecd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a7d90 con 0x557b21ec4240 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a99f0 con 0x557b21ec93b0 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.476+0000 7fd592408740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a8490 con 0x557b21ec3850 2024-03-18T15:48:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41152/0 (socket says 172.21.15.96:41152) 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 -- 172.21.15.96:0/658806491 learned_addr learned my addr 172.21.15.96:0/658806491 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ebf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ecd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 -- 172.21.15.96:0/658806491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 msgr2=0x557b21ebf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ebf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 -- 172.21.15.96:0/658806491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 msgr2=0x557b21ecd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ecd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 -- 172.21.15.96:0/658806491 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b21c067d0 con 0x557b21ec4240 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ecd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd578036040 tx=0x7fd578005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a0d2fd78761228 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd572ffd640 1 -- 172.21.15.96:0/658806491 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57804cb20 con 0x557b21ec4240 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd572ffd640 1 -- 172.21.15.96:0/658806491 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57804c020 con 0x557b21ec4240 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.477+0000 7fd572ffd640 1 -- 172.21.15.96:0/658806491 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578003830 con 0x557b21ec4240 2024-03-18T15:48:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- 172.21.15.96:0/658806491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 msgr2=0x557b21ed50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd578036040 tx=0x7fd578005fd0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- 172.21.15.96:0/658806491 shutdown_connections 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec93b0 0x557b21ecd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec4240 0x557b21ed50e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- 172.21.15.96:0/658806491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ebf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- 172.21.15.96:0/658806491 >> 172.21.15.96:0/658806491 conn(0x557b21ebedc0 msgr2=0x557b21eba0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- 172.21.15.96:0/658806491 shutdown_connections 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- 172.21.15.96:0/658806491 wait complete. 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- start start 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 Processor -- start 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ed24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a9320 con 0x557b21ec93b0 2024-03-18T15:48:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a8b90 con 0x557b21ec4240 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.478+0000 7fd592408740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881a5d70 con 0x557b21ec3850 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd592408740 10 client.0 _ll_register_callbacks cb 0x7fd5840044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58ddb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58cdb4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ed24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42558/0 (socket says 172.21.15.96:42558) 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58ddb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41164/0 (socket says 172.21.15.96:41164) 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 -- 172.21.15.96:0/2974315577 learned_addr learned my addr 172.21.15.96:0/2974315577 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 -- 172.21.15.96:0/2974315577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 msgr2=0x557b21ed24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ed24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 -- 172.21.15.96:0/2974315577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 msgr2=0x557b21f3df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 -- 172.21.15.96:0/2974315577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b21f3e450 con 0x557b21ec3850 2024-03-18T15:48:47.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58ddb6640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ed24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd58d5b5640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd584026620 tx=0x7fd584026950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64cb27c718160428 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd570ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584035070 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57804c020 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fd584026a70 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd584041050 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd592408740 10 client.27613 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.479+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b21f3f720 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57804c020 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7fd58401a040 con 0x557b21ec3850 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 10 client.27613 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 10 client.27613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 20 client.27613 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21f42140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 4 client.27613 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 --> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b21e2fd20 con 0x557b21ed9b40 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21f42140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd570ff9640 10 client.27613 ms_handle_connect on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.480+0000 7fd58cdb4640 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21f42140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf9f1338cad48b39 server_cookie=f70ee7ce8af17cbf in_seq=0 out_seq=0 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd570ff9640 1 -- 172.21.15.96:0/2974315577 <== mds.3 v2:172.21.15.193:6834/1114653826 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b21e2fd20 con 0x557b21ed9b40 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd570ff9640 10 client.27613 handle_command_reply: tid=0 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 client.27613 shutdown 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 msgr2=0x557b21ed1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd584026620 tx=0x7fd584026950 comp rx=0 tx=0).stop 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 shutdown_connections 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:6834/1114653826,v1:172.21.15.193:6835/1114653826] conn(0x557b21ed9b40 0x557b21f42140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557b21ec93b0 0x557b21f3df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd570ff9640 0 client.27613 ms_handle_reset on v2:172.21.15.193:6834/1114653826 2024-03-18T15:48:47.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b21ec4240 0x557b21ed24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 --2- 172.21.15.96:0/2974315577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557b21ec3850 0x557b21ed1f80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 >> 172.21.15.96:0/2974315577 conn(0x557b21ebedc0 msgr2=0x557b21ebf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 shutdown_connections 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 1 -- 172.21.15.96:0/2974315577 wait complete. 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd592408740 20 client.27613 trim_cache size 0 max 16384 2024-03-18T15:48:47.489 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 msgr2=0x7fd5740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 0x7fd5740a4cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd588101730 tx=0x7fd584005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 msgr2=0x7fd58819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd58000eb10 tx=0x7fd58000c2d0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 shutdown_connections 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fd5740a2800 0x7fd5740a4cc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd588102290 0x7fd58819dfb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5881018d0 0x7fd588196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 --2- 172.21.15.96:0/2068466915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5881006e0 0x7fd588196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 >> 172.21.15.96:0/2068466915 conn(0x7fd5880fbc70 msgr2=0x7fd588104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 shutdown_connections 2024-03-18T15:48:47.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.481+0000 7fd58f819640 1 -- 172.21.15.96:0/2068466915 wait complete. 2024-03-18T15:48:47.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 -- 172.21.15.96:0/3567546402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0101800 msgr2=0x7f73e0101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/3567546402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0101800 0x7f73e0101c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009860 tx=0x7f73d002bb70 comp rx=0 tx=0).stop 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 -- 172.21.15.96:0/3567546402 shutdown_connections 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/3567546402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e01033b0 0x7f73e0109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/3567546402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 0x7f73e0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/3567546402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e0101800 0x7f73e0101c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 -- 172.21.15.96:0/3567546402 >> 172.21.15.96:0/3567546402 conn(0x7f73e00fcd70 msgr2=0x7f73e00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 -- 172.21.15.96:0/3567546402 shutdown_connections 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.597+0000 7f73e7e3d640 1 -- 172.21.15.96:0/3567546402 wait complete. 2024-03-18T15:48:47.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 Processor -- start 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 -- start start 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 0x7f73e0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e01033b0 0x7f73e019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d450 con 0x7f73e0101800 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d2d0 con 0x7f73e01029f0 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e7e3d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d5d0 con 0x7f73e01033b0 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41170/0 (socket says 172.21.15.96:41170) 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 -- 172.21.15.96:0/1700742768 learned_addr learned my addr 172.21.15.96:0/1700742768 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e53d8640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 0x7f73e0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e63da640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e01033b0 0x7f73e019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 -- 172.21.15.96:0/1700742768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e01033b0 msgr2=0x7f73e019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e01033b0 0x7f73e019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 -- 172.21.15.96:0/1700742768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 msgr2=0x7f73e0198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 0x7f73e0198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 -- 172.21.15.96:0/1700742768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e019f740 con 0x7f73e0101800 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.598+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f73d002c0b0 tx=0x7f73d0032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.599+0000 7f73d6ffd640 1 -- 172.21.15.96:0/1700742768 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0041020 con 0x7f73e0101800 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.599+0000 7f73d6ffd640 1 -- 172.21.15.96:0/1700742768 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d002c9f0 con 0x7f73e0101800 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.599+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e019f9d0 con 0x7f73e0101800 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.599+0000 7f73d6ffd640 1 -- 172.21.15.96:0/1700742768 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0039040 con 0x7f73e0101800 2024-03-18T15:48:47.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.599+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e019fec0 con 0x7f73e0101800 2024-03-18T15:48:47.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.601+0000 7f73d6ffd640 1 -- 172.21.15.96:0/1700742768 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f73d0044020 con 0x7f73e0101800 2024-03-18T15:48:47.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.601+0000 7f73eaa21740 1 Processor -- start 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 -- start start 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee81a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73d6ffd640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 0x7f73cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a77c0 0x55eee81a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eee8102490 con 0x55eee81a2650 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eee80aac90 con 0x55eee81a1c60 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73eaa21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eee80fe530 con 0x55eee81a77c0 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e53d8640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 0x7f73cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41186/0 (socket says 172.21.15.96:41186) 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e63da640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee81a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 -- 172.21.15.96:0/710408003 learned_addr learned my addr 172.21.15.96:0/710408003 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e63da640 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a77c0 0x55eee81a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73d6ffd640 1 -- 172.21.15.96:0/1700742768 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f73d00f0360 con 0x7f73e0101800 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 -- 172.21.15.96:0/710408003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a77c0 msgr2=0x55eee81a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a77c0 0x55eee81a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 -- 172.21.15.96:0/710408003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 msgr2=0x55eee81a2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee81a2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 -- 172.21.15.96:0/710408003 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee8165500 con 0x55eee81a2650 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e63da640 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee81a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e53d8640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 0x7f73cc0a4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0095f0 tx=0x7f73dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.602+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b9130 tx=0x7f73d008bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5bc3d79dbee4a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73cbfff640 1 -- 172.21.15.96:0/710408003 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00b8f70 con 0x55eee81a2650 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73cbfff640 1 -- 172.21.15.96:0/710408003 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00411c0 con 0x55eee81a2650 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73cbfff640 1 -- 172.21.15.96:0/710408003 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0086050 con 0x55eee81a2650 2024-03-18T15:48:47.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 -- 172.21.15.96:0/710408003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 msgr2=0x55eee81a7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f73d00b9130 tx=0x7f73d008bb30 comp rx=0 tx=0).stop 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 -- 172.21.15.96:0/710408003 shutdown_connections 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a77c0 0x55eee81a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81a7150 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 --2- 172.21.15.96:0/710408003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee81a2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.603+0000 7f73eaa21740 1 -- 172.21.15.96:0/710408003 >> 172.21.15.96:0/710408003 conn(0x55eee8198dc0 msgr2=0x55eee81991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- 172.21.15.96:0/710408003 shutdown_connections 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- 172.21.15.96:0/710408003 wait complete. 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- start start 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 Processor -- start 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a1c60 0x55eee81abea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee81b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eee81aa530 con 0x55eee81a2650 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eee80be230 con 0x55eee81a77c0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eee81aa6b0 con 0x55eee81a1c60 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73eaa21740 10 client.0 _ll_register_callbacks cb 0x55eee80fa010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73e53d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73e53d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41200/0 (socket says 172.21.15.96:41200) 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73e53d8640 1 -- 172.21.15.96:0/994528033 learned_addr learned my addr 172.21.15.96:0/994528033 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a1c60 0x55eee81abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.604+0000 7f73e63da640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee81b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 -- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a1c60 msgr2=0x55eee81abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a1c60 0x55eee81abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 -- 172.21.15.96:0/994528033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 msgr2=0x55eee81b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee81b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 -- 172.21.15.96:0/994528033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee80cd100 con 0x55eee81a2650 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73e53d8640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f73dc003850 tx=0x7f73dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e83364b16f5999 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00eab0 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f73dc00ec50 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eee810f9d0 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f73dc00ee50 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 10 client.17799 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55eee7e163f0 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55eee810f9d0 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f73dc02f070 con 0x55eee81a2650 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 10 client.17799 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 10 client.17799 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 20 client.17799 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee81b7e60 0x55eee81ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 4 client.17799 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.605+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee8109d20 con 0x55eee81b7e60 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.606+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee81b7e60 0x55eee81ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.606+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee81b7e60 0x55eee81ba2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de57d0abb52dca47 server_cookie=455aeacc126f90b5 in_seq=0 out_seq=0 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.606+0000 7f73c97fa640 10 client.17799 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73c97fa640 1 -- 172.21.15.96:0/994528033 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55eee8109d20 con 0x55eee81b7e60 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73c97fa640 10 client.17799 handle_command_reply: tid=0 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 client.17799 shutdown 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 msgr2=0x55eee81ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f73dc003850 tx=0x7f73dc004730 comp rx=0 tx=0).stop 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 shutdown_connections 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee81b7e60 0x55eee81ba2a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee81b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a2650 0x55eee81ac3e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 --2- 172.21.15.96:0/994528033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a1c60 0x55eee81abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73c97fa640 0 client.17799 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:47.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 >> 172.21.15.96:0/994528033 conn(0x55eee8198dc0 msgr2=0x55eee8197fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 shutdown_connections 2024-03-18T15:48:47.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 1 -- 172.21.15.96:0/994528033 wait complete. 2024-03-18T15:48:47.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.607+0000 7f73eaa21740 20 client.17799 trim_cache size 0 max 16384 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.620+0000 7f73eaa21740 1 Processor -- start 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.620+0000 7f73eaa21740 1 -- start start 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee8197750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a6c00 con 0x55eee81a77c0 2024-03-18T15:48:47.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a8860 con 0x55eee81a1c60 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73eaa21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a7300 con 0x55eee81a2650 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41210/0 (socket says 172.21.15.96:41210) 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 -- 172.21.15.96:0/4260797510 learned_addr learned my addr 172.21.15.96:0/4260797510 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e53d8640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 -- 172.21.15.96:0/4260797510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 msgr2=0x55eee81b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 -- 172.21.15.96:0/4260797510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 msgr2=0x55eee8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 -- 172.21.15.96:0/4260797510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee7ee07d0 con 0x55eee81a77c0 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e53d8640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73e63da640 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f73d800eaa0 tx=0x7f73d8010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b5b83cb9d245c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73cb7fe640 1 -- 172.21.15.96:0/4260797510 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8021070 con 0x55eee81a77c0 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73cb7fe640 1 -- 172.21.15.96:0/4260797510 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d800dc70 con 0x55eee81a77c0 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.621+0000 7f73cb7fe640 1 -- 172.21.15.96:0/4260797510 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d801d450 con 0x55eee81a77c0 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- 172.21.15.96:0/4260797510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 msgr2=0x55eee8197c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f73d800eaa0 tx=0x7f73d8010040 comp rx=0 tx=0).stop 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- 172.21.15.96:0/4260797510 shutdown_connections 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a77c0 0x55eee8197c90 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 --2- 172.21.15.96:0/4260797510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a1c60 0x55eee8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- 172.21.15.96:0/4260797510 >> 172.21.15.96:0/4260797510 conn(0x55eee8198dc0 msgr2=0x55eee81a3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- 172.21.15.96:0/4260797510 shutdown_connections 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- 172.21.15.96:0/4260797510 wait complete. 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 -- start start 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.622+0000 7f73eaa21740 1 Processor -- start 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee821c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a8190 con 0x55eee81a1c60 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a7a00 con 0x55eee81a77c0 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a4be0 con 0x55eee81a2650 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 10 client.0 _ll_register_callbacks cb 0x7f73dc005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41224/0 (socket says 172.21.15.96:41224) 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 -- 172.21.15.96:0/2232710327 learned_addr learned my addr 172.21.15.96:0/2232710327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e63da640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee821c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e53d8640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 -- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 msgr2=0x55eee81b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 -- 172.21.15.96:0/2232710327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 msgr2=0x55eee821c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee821c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 -- 172.21.15.96:0/2232710327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee821c700 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73e5bd9640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f73d008f1f0 tx=0x7f73d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad26e21072ee8912 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d00bc630 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f73d0086050 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d800dc70 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f73d00d5030 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 10 client.17805 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.623+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55eee821dab0 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d800dc70 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f73d00ef070 con 0x55eee81a1c60 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 10 client.17805 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 10 client.17805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 20 client.17805 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee821e370 0x55eee8220790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 4 client.17805 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 --> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee80876a0 con 0x55eee821e370 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73e53d8640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee821e370 0x55eee8220790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73e53d8640 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee821e370 0x55eee8220790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31bfe05a11459e58 server_cookie=6d7d265930ce5947 in_seq=0 out_seq=0 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.624+0000 7f73c97fa640 10 client.17805 ms_handle_connect on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73c97fa640 1 -- 172.21.15.96:0/2232710327 <== mds.2 v2:172.21.15.193:6836/2478672579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee80876a0 con 0x55eee821e370 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73c97fa640 10 client.17805 handle_command_reply: tid=0 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 client.17805 shutdown 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 msgr2=0x55eee81b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f73d008f1f0 tx=0x7f73d0005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 shutdown_connections 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:6836/2478672579,v1:172.21.15.193:6837/2478672579] conn(0x55eee821e370 0x55eee8220790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eee81a77c0 0x55eee821c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73c97fa640 0 client.17805 ms_handle_reset on v2:172.21.15.193:6836/2478672579 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eee81a2650 0x55eee81b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 --2- 172.21.15.96:0/2232710327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee81a1c60 0x55eee81b0420 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 >> 172.21.15.96:0/2232710327 conn(0x55eee8198dc0 msgr2=0x55eee819c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 shutdown_connections 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 1 -- 172.21.15.96:0/2232710327 wait complete. 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73eaa21740 20 client.17805 trim_cache size 0 max 16384 2024-03-18T15:48:47.633 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 msgr2=0x7f73cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 0x7f73cc0a4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0095f0 tx=0x7f73dc005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 msgr2=0x7f73e0197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f73d002c0b0 tx=0x7f73d0032430 comp rx=0 tx=0).stop 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 shutdown_connections 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f73cc0a2800 0x7f73cc0a4cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e01033b0 0x7f73e019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f73e01029f0 0x7f73e0198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.625+0000 7f73e7e3d640 1 --2- 172.21.15.96:0/1700742768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73e0101800 0x7f73e0197b20 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.626+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 >> 172.21.15.96:0/1700742768 conn(0x7f73e00fcd70 msgr2=0x7f73e0105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.626+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 shutdown_connections 2024-03-18T15:48:47.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.626+0000 7f73e7e3d640 1 -- 172.21.15.96:0/1700742768 wait complete. 2024-03-18T15:48:47.637 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- 172.21.15.96:0/3116477976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 msgr2=0x7f7d24109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 --2- 172.21.15.96:0/3116477976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d24109c30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ca0 tx=0x7f7d1c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- 172.21.15.96:0/3116477976 shutdown_connections 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 --2- 172.21.15.96:0/3116477976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d24109c30 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 --2- 172.21.15.96:0/3116477976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d24102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 --2- 172.21.15.96:0/3116477976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 0x7f7d24101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- 172.21.15.96:0/3116477976 >> 172.21.15.96:0/3116477976 conn(0x7f7d240fcd80 msgr2=0x7f7d240ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- 172.21.15.96:0/3116477976 shutdown_connections 2024-03-18T15:48:47.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- 172.21.15.96:0/3116477976 wait complete. 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 Processor -- start 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.741+0000 7f7d29945640 1 -- start start 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 0x7f7d24199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d2419a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d470 con 0x7f7d241033a0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d2f0 con 0x7f7d241017f0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d29945640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410d5f0 con 0x7f7d241029e0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41248/0 (socket says 172.21.15.96:41248) 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 -- 172.21.15.96:0/1316733494 learned_addr learned my addr 172.21.15.96:0/1316733494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d2419a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 0x7f7d24199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 -- 172.21.15.96:0/1316733494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 msgr2=0x7f7d2419a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d2419a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 -- 172.21.15.96:0/1316733494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 msgr2=0x7f7d24199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 0x7f7d24199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 -- 172.21.15.96:0/1316733494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d241a1920 con 0x7f7d241033a0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d2419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.742+0000 7f7d237fe640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c032530 tx=0x7f7d1c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.743+0000 7f7d28943640 1 -- 172.21.15.96:0/1316733494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c041020 con 0x7f7d241033a0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.743+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241a1bb0 con 0x7f7d241033a0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.743+0000 7f7d28943640 1 -- 172.21.15.96:0/1316733494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c034cd0 con 0x7f7d241033a0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.743+0000 7f7d28943640 1 -- 172.21.15.96:0/1316733494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c039040 con 0x7f7d241033a0 2024-03-18T15:48:47.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.743+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241a2070 con 0x7f7d241033a0 2024-03-18T15:48:47.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.745+0000 7f7d28943640 1 -- 172.21.15.96:0/1316733494 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f7d1c044020 con 0x7f7d241033a0 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.745+0000 7f7d2c60d740 1 Processor -- start 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 -- start start 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccba71a490 con 0x55ccba7bb3b0 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ccba6c2c90 con 0x55ccba7b5850 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ccba716530 con 0x55ccba7b6240 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41258/0 (socket says 172.21.15.96:41258) 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d22ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1839721515 learned_addr learned my addr 172.21.15.96:0/1839721515 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d28943640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 0x7f7d100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1839721515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 msgr2=0x55ccba7bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1839721515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 msgr2=0x55ccba7b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1839721515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccba77d500 con 0x55ccba7bb3b0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d237fe640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 0x7f7d100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d28943640 1 -- 172.21.15.96:0/1316733494 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7d1c0f0940 con 0x7f7d241033a0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.746+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009710 tx=0x7f7d0c009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15808618fbfb7e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d19ffb640 1 -- 172.21.15.96:0/1839721515 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00fc50 con 0x55ccba7bb3b0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d19ffb640 1 -- 172.21.15.96:0/1839721515 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c00fdf0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d19ffb640 1 -- 172.21.15.96:0/1839721515 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00e6d0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1839721515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 msgr2=0x55ccba7bd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d237fe640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 0x7f7d100a4cc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0bcfb0 tx=0x7f7d1c087fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009710 tx=0x7f7d0c009bc0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1839721515 shutdown_connections 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7bd840 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1839721515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1839721515 >> 172.21.15.96:0/1839721515 conn(0x55ccba7b0dc0 msgr2=0x55ccba7b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1839721515 shutdown_connections 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.747+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1839721515 wait complete. 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 -- start start 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 Processor -- start 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ccba7be360 con 0x55ccba7bb3b0 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ccba6d6230 con 0x55ccba7b5850 2024-03-18T15:48:47.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ccba7be4e0 con 0x55ccba7b6240 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d2c60d740 10 client.0 _ll_register_callbacks cb 0x55ccba712010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41264/0 (socket says 172.21.15.96:41264) 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1672999525 learned_addr learned my addr 172.21.15.96:0/1672999525 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1672999525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 msgr2=0x55ccba7c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 msgr2=0x55ccba7bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.748+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/1672999525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccba6e5100 con 0x55ccba7bb3b0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c001eb0 tx=0x7f7d0c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1564621c8edf63 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c005330 con 0x55ccba7bb3b0 2024-03-18T15:48:47.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7d0c028040 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c00fdf0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c0054d0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 10 client.17817 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ccba42e3f0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c00fdf0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f7d0c036020 con 0x55ccba7bb3b0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 10 client.17817 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 10 client.17817 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 20 client.17817 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba7cb930 0x55ccba7cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 4 client.17817 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.749+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccba7279d0 con 0x55ccba7cb930 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.750+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba7cb930 0x55ccba7cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.751+0000 7f7d02ffd640 10 client.17817 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:47.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.751+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba7cb930 0x55ccba7cdd80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d6d7b5467c339f2 server_cookie=b07a720e2bc4e442 in_seq=0 out_seq=0 2024-03-18T15:48:47.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1672999525 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55ccba7279d0 con 0x55ccba7cb930 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d02ffd640 10 client.17817 handle_command_reply: tid=0 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 client.17817 shutdown 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 msgr2=0x55ccba7c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c001eb0 tx=0x7f7d0c005850 comp rx=0 tx=0).stop 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 shutdown_connections 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7330 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba7cb930 0x55ccba7cdd80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1672999525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 >> 172.21.15.96:0/1672999525 conn(0x55ccba7b0dc0 msgr2=0x55ccba7affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d02ffd640 0 client.17817 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 shutdown_connections 2024-03-18T15:48:47.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1672999525 wait complete. 2024-03-18T15:48:47.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.752+0000 7f7d2c60d740 20 client.17817 trim_cache size 0 max 16384 2024-03-18T15:48:47.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 Processor -- start 2024-03-18T15:48:47.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 -- start start 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241aaf90 con 0x55ccba7b5850 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241acbf0 con 0x55ccba7bb3b0 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241ab690 con 0x55ccba7b6240 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41274/0 (socket says 172.21.15.96:41274) 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/4143608172 learned_addr learned my addr 172.21.15.96:0/4143608172 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.765+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/4143608172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 msgr2=0x55ccba7c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/4143608172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7bb3b0 msgr2=0x55ccba7c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 -- 172.21.15.96:0/4143608172 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccba4f87d0 con 0x55ccba7b5850 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c011970 tx=0x7f7d0c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290a9688c8180a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d197fa640 1 -- 172.21.15.96:0/4143608172 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0052e0 con 0x55ccba7b5850 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d197fa640 1 -- 172.21.15.96:0/4143608172 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c033070 con 0x55ccba7b5850 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d197fa640 1 -- 172.21.15.96:0/4143608172 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c004f50 con 0x55ccba7b5850 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 -- 172.21.15.96:0/4143608172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 msgr2=0x55ccba7bfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c011970 tx=0x7f7d0c005670 comp rx=0 tx=0).stop 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 -- 172.21.15.96:0/4143608172 shutdown_connections 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7b6240 0x55ccba7c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/4143608172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b5850 0x55ccba7bfdf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.766+0000 7f7d2c60d740 1 -- 172.21.15.96:0/4143608172 >> 172.21.15.96:0/4143608172 conn(0x55ccba7b0dc0 msgr2=0x55ccba7b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- 172.21.15.96:0/4143608172 shutdown_connections 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- 172.21.15.96:0/4143608172 wait complete. 2024-03-18T15:48:47.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- start start 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 Processor -- start 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241ac520 con 0x55ccba7b6240 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241abd90 con 0x55ccba7b5850 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241a8f70 con 0x55ccba7bb3b0 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d2c60d740 10 client.0 _ll_register_callbacks cb 0x7f7d0c011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d1a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d22ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41290/0 (socket says 172.21.15.96:41290) 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d22ffd640 1 -- 172.21.15.96:0/1689573116 learned_addr learned my addr 172.21.15.96:0/1689573116 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.767+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 -- 172.21.15.96:0/1689573116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7bb3b0 msgr2=0x55ccba7c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 -- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 msgr2=0x55ccba7c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 -- 172.21.15.96:0/1689573116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccba82fcf0 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d22ffd640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x55ccba7b6cb0 tx=0x7f7d140206f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdd2bb0a34af8f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140037f0 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d0c033070 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f7d14003990 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7d14003b90 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 10 client.17823 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ccba8310c0 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d0c033070 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f7d14032070 con 0x55ccba7b6240 2024-03-18T15:48:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 10 client.17823 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 10 client.17823 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 20 client.17823 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba831a40 0x55ccba833e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 4 client.17823 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.768+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 --> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccba721d20 con 0x55ccba831a40 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba831a40 0x55ccba833e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d1a7fc640 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba831a40 0x55ccba833e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38b952023a89b9b5 server_cookie=c13da23ce5d931b1 in_seq=0 out_seq=0 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d02ffd640 10 client.17823 ms_handle_connect on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d02ffd640 1 -- 172.21.15.96:0/1689573116 <== mds.1 v2:172.21.15.137:6838/1070718965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccba721d20 con 0x55ccba831a40 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d02ffd640 10 client.17823 handle_command_reply: tid=0 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 client.17823 shutdown 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 msgr2=0x55ccba7c3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x55ccba7b6cb0 tx=0x7f7d140206f0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 shutdown_connections 2024-03-18T15:48:47.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ccba7bb3b0 0x55ccba7c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ccba7b6240 0x55ccba7c3db0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:6838/1070718965,v1:172.21.15.137:6839/1070718965] conn(0x55ccba831a40 0x55ccba833e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 --2- 172.21.15.96:0/1689573116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccba7b5850 0x55ccba7c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d02ffd640 0 client.17823 ms_handle_reset on v2:172.21.15.137:6838/1070718965 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 >> 172.21.15.96:0/1689573116 conn(0x55ccba7b0dc0 msgr2=0x55ccba7b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 shutdown_connections 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 1 -- 172.21.15.96:0/1689573116 wait complete. 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.769+0000 7f7d2c60d740 20 client.17823 trim_cache size 0 max 16384 2024-03-18T15:48:47.777 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 msgr2=0x7f7d100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 0x7f7d100a4cc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0bcfb0 tx=0x7f7d1c087fe0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 msgr2=0x7f7d241a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c032530 tx=0x7f7d1c032f20 comp rx=0 tx=0).stop 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 shutdown_connections 2024-03-18T15:48:47.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f7d100a2800 0x7f7d100a4cc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d241033a0 0x7f7d241a1280 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d241029e0 0x7f7d2419a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 --2- 172.21.15.96:0/1316733494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d241017f0 0x7f7d24199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 >> 172.21.15.96:0/1316733494 conn(0x7f7d240fcd80 msgr2=0x7f7d24073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 shutdown_connections 2024-03-18T15:48:47.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.770+0000 7f7d29945640 1 -- 172.21.15.96:0/1316733494 wait complete. 2024-03-18T15:48:47.780 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:47 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3644912792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:47.782 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-18T15:48:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 -- 172.21.15.96:0/3504446563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581029e0 msgr2=0x7f0958102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 --2- 172.21.15.96:0/3504446563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581029e0 0x7f0958102e60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f094802bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:47.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 -- 172.21.15.96:0/3504446563 shutdown_connections 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 --2- 172.21.15.96:0/3504446563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581033a0 0x7f0958109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 --2- 172.21.15.96:0/3504446563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581029e0 0x7f0958102e60 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 --2- 172.21.15.96:0/3504446563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 -- 172.21.15.96:0/3504446563 >> 172.21.15.96:0/3504446563 conn(0x7f09580fcd80 msgr2=0x7f09580ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 -- 172.21.15.96:0/3504446563 shutdown_connections 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.885+0000 7f095f212640 1 -- 172.21.15.96:0/3504446563 wait complete. 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 Processor -- start 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 -- start start 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581029e0 0x7f0958198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f095810d470 con 0x7f09581033a0 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f095810d2f0 con 0x7f09581029e0 2024-03-18T15:48:47.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095f212640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f095810d5f0 con 0x7f09581017f0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095cfae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41300/0 (socket says 172.21.15.96:41300) 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 -- 172.21.15.96:0/1363382812 learned_addr learned my addr 172.21.15.96:0/1363382812 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f094ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581029e0 0x7f0958198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095cfae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42682/0 (socket says 172.21.15.96:42682) 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 -- 172.21.15.96:0/1363382812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 msgr2=0x7f0958197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 -- 172.21.15.96:0/1363382812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581029e0 msgr2=0x7f0958198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581029e0 0x7f0958198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.886+0000 7f095d7af640 1 -- 172.21.15.96:0/1363382812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095819f740 con 0x7f09581033a0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f095d7af640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f095000efd0 tx=0x7f095000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f094dffb640 1 -- 172.21.15.96:0/1363382812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950003b70 con 0x7f09581033a0 2024-03-18T15:48:47.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095819fa30 con 0x7f09581033a0 2024-03-18T15:48:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f094dffb640 1 -- 172.21.15.96:0/1363382812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0950003d10 con 0x7f09581033a0 2024-03-18T15:48:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f094dffb640 1 -- 172.21.15.96:0/1363382812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500109f0 con 0x7f09581033a0 2024-03-18T15:48:47.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.887+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095819ff00 con 0x7f09581033a0 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f094dffb640 1 -- 172.21.15.96:0/1363382812 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0950004020 con 0x7f09581033a0 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f094dffb640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 0x7f09400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f0961e00740 1 Processor -- start 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f0961e00740 1 -- start start 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f094dffb640 1 -- 172.21.15.96:0/1363382812 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f09500cee90 con 0x7f09581033a0 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f095cfae640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 0x7f09400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f0961e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.889+0000 7f0961e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d10150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f0961e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d12c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f0961e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5599b4c6b490 con 0x5599b4d0ac60 2024-03-18T15:48:47.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f0961e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599b4c13c90 con 0x5599b4d107c0 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f0961e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599b4c67530 con 0x5599b4d0b650 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41308/0 (socket says 172.21.15.96:41308) 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 -- 172.21.15.96:0/549677916 learned_addr learned my addr 172.21.15.96:0/549677916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f094ffff640 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d10150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095d7af640 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d12c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 0x7f09400a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0954013570 tx=0x7f0954013970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 -- 172.21.15.96:0/549677916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 msgr2=0x5599b4d10150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d10150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 -- 172.21.15.96:0/549677916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 msgr2=0x5599b4d12c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d12c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 -- 172.21.15.96:0/549677916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b4cce500 con 0x5599b4d0ac60 2024-03-18T15:48:47.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f095cfae640 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f095400f8c0 tx=0x7f0954020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5f2bfb2707379d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f093effd640 1 -- 172.21.15.96:0/549677916 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004020 con 0x5599b4d0ac60 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f093effd640 1 -- 172.21.15.96:0/549677916 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0954024070 con 0x5599b4d0ac60 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.890+0000 7f093effd640 1 -- 172.21.15.96:0/549677916 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954003850 con 0x5599b4d0ac60 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- 172.21.15.96:0/549677916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 msgr2=0x5599b4d0b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f095400f8c0 tx=0x7f0954020000 comp rx=0 tx=0).stop 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- 172.21.15.96:0/549677916 shutdown_connections 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d12c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d10150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 --2- 172.21.15.96:0/549677916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d0b080 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- 172.21.15.96:0/549677916 >> 172.21.15.96:0/549677916 conn(0x5599b4d01dc0 msgr2=0x5599b4d021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- 172.21.15.96:0/549677916 shutdown_connections 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- 172.21.15.96:0/549677916 wait complete. 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 -- start start 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.891+0000 7f0961e00740 1 Processor -- start 2024-03-18T15:48:47.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d15600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5599b4d13710 con 0x5599b4d0ac60 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599b4c27230 con 0x5599b4d107c0 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5599b4d13890 con 0x5599b4d0b650 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 10 client.0 _ll_register_callbacks cb 0x5599b4c63010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41316/0 (socket says 172.21.15.96:41316) 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 -- 172.21.15.96:0/3293031216 learned_addr learned my addr 172.21.15.96:0/3293031216 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f094ffff640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095d7af640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 -- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 msgr2=0x5599b4d15600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d15600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 -- 172.21.15.96:0/3293031216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 msgr2=0x5599b4d1c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d1c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 -- 172.21.15.96:0/3293031216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b4c36100 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095d7af640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d1c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f095cfae640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0954009b70 tx=0x7f0954020b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6825857f50ea34 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0937fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954005250 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f09540053f0 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.892+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0954024070 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0954038030 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 10 client.17835 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5599b497f3f0 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0954024070 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f095401f070 con 0x5599b4d0ac60 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 10 client.17835 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 20 client.17835 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d20ca0 0x5599b4d23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 4 client.17835 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599b4c789d0 con 0x5599b4d20ca0 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.893+0000 7f094ffff640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d20ca0 0x5599b4d23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.894+0000 7f0937fff640 10 client.17835 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.894+0000 7f094ffff640 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d20ca0 0x5599b4d23090 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afbc5121a9f659a2 server_cookie=be7167ed14f963b1 in_seq=0 out_seq=0 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0937fff640 1 -- 172.21.15.96:0/3293031216 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x5599b4c789d0 con 0x5599b4d20ca0 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0937fff640 10 client.17835 handle_command_reply: tid=0 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 client.17835 shutdown 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 msgr2=0x5599b4d150c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0954009b70 tx=0x7f0954020b60 comp rx=0 tx=0).stop 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 shutdown_connections 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d1c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d15600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d20ca0 0x5599b4d23090 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 --2- 172.21.15.96:0/3293031216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0ac60 0x5599b4d150c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0937fff640 0 client.17835 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 >> 172.21.15.96:0/3293031216 conn(0x5599b4d01dc0 msgr2=0x5599b4d00fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 shutdown_connections 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 1 -- 172.21.15.96:0/3293031216 wait complete. 2024-03-18T15:48:47.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.895+0000 7f0961e00740 20 client.17835 trim_cache size 0 max 16384 2024-03-18T15:48:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 Processor -- start 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 -- start start 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0ac60 0x5599b4d15340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a6b50 con 0x5599b4d0b650 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a87b0 con 0x5599b4d107c0 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f0961e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a7250 con 0x5599b4d0ac60 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41326/0 (socket says 172.21.15.96:41326) 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 -- 172.21.15.96:0/1497152786 learned_addr learned my addr 172.21.15.96:0/1497152786 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f095cfae640 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0ac60 0x5599b4d15340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f095d7af640 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 -- 172.21.15.96:0/1497152786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0ac60 msgr2=0x5599b4d15340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0ac60 0x5599b4d15340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 -- 172.21.15.96:0/1497152786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 msgr2=0x5599b4d278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.908+0000 7f094ffff640 1 -- 172.21.15.96:0/1497152786 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599b4d23400 con 0x5599b4d0b650 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f094ffff640 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0948009850 tx=0x7f0948005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb140e235741ad0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f093e7fc640 1 -- 172.21.15.96:0/1497152786 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094802cd20 con 0x5599b4d0b650 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f093e7fc640 1 -- 172.21.15.96:0/1497152786 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0948004040 con 0x5599b4d0b650 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f093e7fc640 1 -- 172.21.15.96:0/1497152786 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948005040 con 0x5599b4d0b650 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 -- 172.21.15.96:0/1497152786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 msgr2=0x5599b4d1c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0948009850 tx=0x7f0948005fd0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 -- 172.21.15.96:0/1497152786 shutdown_connections 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d107c0 0x5599b4d278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d0b650 0x5599b4d1c600 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 --2- 172.21.15.96:0/1497152786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0ac60 0x5599b4d15340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 -- 172.21.15.96:0/1497152786 >> 172.21.15.96:0/1497152786 conn(0x5599b4d01dc0 msgr2=0x5599b4d058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 -- 172.21.15.96:0/1497152786 shutdown_connections 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.909+0000 7f0961e00740 1 -- 172.21.15.96:0/1497152786 wait complete. 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 -- start start 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 Processor -- start 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d0ac60 0x5599b4d19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d1a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a80e0 con 0x5599b4d107c0 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a7950 con 0x5599b4d0ac60 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09581a4b30 con 0x5599b4d0b650 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f0961e00740 10 client.0 _ll_register_callbacks cb 0x7f0950009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41332/0 (socket says 172.21.15.96:41332) 2024-03-18T15:48:47.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 -- 172.21.15.96:0/4190780800 learned_addr learned my addr 172.21.15.96:0/4190780800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f094ffff640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d1a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095cfae640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d0ac60 0x5599b4d19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 -- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 msgr2=0x5599b4d1a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d1a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 -- 172.21.15.96:0/4190780800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d0ac60 msgr2=0x5599b4d19f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d0ac60 0x5599b4d19f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.910+0000 7f095d7af640 1 -- 172.21.15.96:0/4190780800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599b4a497d0 con 0x5599b4d107c0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f095d7af640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f095000cba0 tx=0x7f09500104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d02ba42a823277 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095009f950 con 0x5599b4d107c0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0950004620 con 0x5599b4d107c0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0948004040 con 0x5599b4d107c0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f09500a3fa0 con 0x5599b4d107c0 2024-03-18T15:48:47.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 10 client.17841 fetch_fsmap learned FSMap version 44 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5599b4d86ff0 con 0x5599b4d107c0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0948004040 con 0x5599b4d107c0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f095001f020 con 0x5599b4d107c0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 10 client.17841 fetch_fsmap finished waiting for FSMap version 44 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 20 client.17841 populate_metadata read hostname 'smithi096' 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d87a80 0x5599b4d89ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 4 client.17841 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 --> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599b4c72d20 con 0x5599b4d87a80 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.911+0000 7f095cfae640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d87a80 0x5599b4d89ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f095cfae640 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d87a80 0x5599b4d89ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d1cb2ab79ee078 server_cookie=4d8032f70124767e in_seq=0 out_seq=0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0937fff640 10 client.17841 ms_handle_connect on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0937fff640 1 -- 172.21.15.96:0/4190780800 <== mds.0 v2:172.21.15.193:6838/1184009925 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599b4c72d20 con 0x5599b4d87a80 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0937fff640 10 client.17841 handle_command_reply: tid=0 2024-03-18T15:48:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 client.17841 shutdown 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 msgr2=0x5599b4d1b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f095000cba0 tx=0x7f09500104c0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 shutdown_connections 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:6838/1184009925,v1:172.21.15.193:6839/1184009925] conn(0x5599b4d87a80 0x5599b4d89ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599b4d107c0 0x5599b4d1b480 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5599b4d0b650 0x5599b4d1a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0937fff640 0 client.17841 ms_handle_reset on v2:172.21.15.193:6838/1184009925 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 --2- 172.21.15.96:0/4190780800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599b4d0ac60 0x5599b4d19f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 >> 172.21.15.96:0/4190780800 conn(0x5599b4d01dc0 msgr2=0x5599b4d058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 shutdown_connections 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 1 -- 172.21.15.96:0/4190780800 wait complete. 2024-03-18T15:48:47.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.912+0000 7f0961e00740 20 client.17841 trim_cache size 0 max 16384 2024-03-18T15:48:47.921 INFO:teuthology.orchestra.run.smithi096.stdout:[] 2024-03-18T15:48:47.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 msgr2=0x7f09400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 0x7f09400a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0954013570 tx=0x7f0954013970 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 msgr2=0x7f095819f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f095000efd0 tx=0x7f095000c6a0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 shutdown_connections 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f09400a2800 0x7f09400a4cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09581033a0 0x7f095819f0d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09581029e0 0x7f0958198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 --2- 172.21.15.96:0/1363382812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09581017f0 0x7f0958197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 >> 172.21.15.96:0/1363382812 conn(0x7f09580fcd80 msgr2=0x7f0958105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 shutdown_connections 2024-03-18T15:48:47.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:47.913+0000 7f095f212640 1 -- 172.21.15.96:0/1363382812 wait complete. 2024-03-18T15:48:48.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:47 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3644912792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:48.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:47 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3644912792' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:48:48.925 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-18T15:48:48.925 INFO:tasks.fwd_scrub:done joining 2024-03-18T15:48:48.926 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-18T15:48:48.937 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:48:48.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:48 smithi096 ceph-mon[31930]: pgmap v614: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- 172.21.15.96:0/2638467522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 msgr2=0x7fc58c108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 --2- 172.21.15.96:0/2638467522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c108b40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc584009900 tx=0x7fc58402bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- 172.21.15.96:0/2638467522 shutdown_connections 2024-03-18T15:48:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 --2- 172.21.15.96:0/2638467522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c108b40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 --2- 172.21.15.96:0/2638467522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 --2- 172.21.15.96:0/2638467522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c100700 0x7fc58c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- 172.21.15.96:0/2638467522 >> 172.21.15.96:0/2638467522 conn(0x7fc58c0fbc70 msgr2=0x7fc58c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- 172.21.15.96:0/2638467522 shutdown_connections 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- 172.21.15.96:0/2638467522 wait complete. 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 Processor -- start 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.049+0000 7fc5941ab640 1 -- start start 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c100700 0x7fc58c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10c380 con 0x7fc58c1022b0 2024-03-18T15:48:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10c200 con 0x7fc58c1018f0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10c500 con 0x7fc58c100700 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41344/0 (socket says 172.21.15.96:41344) 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 -- 172.21.15.96:0/3663957697 learned_addr learned my addr 172.21.15.96:0/3663957697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc591746640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 -- 172.21.15.96:0/3663957697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c100700 msgr2=0x7fc58c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c100700 0x7fc58c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 -- 172.21.15.96:0/3663957697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 msgr2=0x7fc58c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 -- 172.21.15.96:0/3663957697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c19f700 con 0x7fc58c1022b0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc591746640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc592748640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc584005ab0 tx=0x7fc58402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584041070 con 0x7fc58c1022b0 2024-03-18T15:48:49.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c19f990 con 0x7fc58c1022b0 2024-03-18T15:48:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc584004450 con 0x7fc58c1022b0 2024-03-18T15:48:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584003e60 con 0x7fc58c1022b0 2024-03-18T15:48:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.050+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c19fe80 con 0x7fc58c1022b0 2024-03-18T15:48:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.051+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c1a00b0 con 0x7fc58c1022b0 2024-03-18T15:48:49.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.053+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc584044070 con 0x7fc58c1022b0 2024-03-18T15:48:49.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.053+0000 7fc582ffd640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 0x7fc5780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.053+0000 7fc591f47640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 0x7fc5780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.053+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc5840f0b40 con 0x7fc58c1022b0 2024-03-18T15:48:49.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.054+0000 7fc591f47640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 0x7fc5780a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc588005c40 tx=0x7fc588005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.057+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc5840bdbb0 con 0x7fc58c1022b0 2024-03-18T15:48:49.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:48 smithi137 ceph-mon[35868]: pgmap v614: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:49.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:48 smithi193 ceph-mon[36702]: pgmap v614: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.232+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc58c1a0280 con 0x7fc58c1022b0 2024-03-18T15:48:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.233+0000 7fc582ffd640 1 -- 172.21.15.96:0/3663957697 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22715 (secure 0 0 0) 0x7fc5840bd4b0 con 0x7fc58c1022b0 2024-03-18T15:48:49.234 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:48:49.235 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":91,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:46:48.700289+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 msgr2=0x7fc5780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 0x7fc5780a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc588005c40 tx=0x7fc588005bd0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 msgr2=0x7fc58c19f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc584005ab0 tx=0x7fc58402c6c0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 shutdown_connections 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc5780a2800 0x7fc5780a4cc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc58c1022b0 0x7fc58c19f090 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc58c1018f0 0x7fc58c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 --2- 172.21.15.96:0/3663957697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc58c100700 0x7fc58c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 >> 172.21.15.96:0/3663957697 conn(0x7fc58c0fbc70 msgr2=0x7fc58c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 shutdown_connections 2024-03-18T15:48:49.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.235+0000 7fc5941ab640 1 -- 172.21.15.96:0/3663957697 wait complete. 2024-03-18T15:48:49.248 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:48:49.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 -- 172.21.15.96:0/1625880399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1006f0 msgr2=0x7f265c100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 --2- 172.21.15.96:0/1625880399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1006f0 0x7f265c100b10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f264c009900 tx=0x7f264c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 -- 172.21.15.96:0/1625880399 shutdown_connections 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 --2- 172.21.15.96:0/1625880399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1022a0 0x7f265c108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 --2- 172.21.15.96:0/1625880399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1018e0 0x7f265c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 --2- 172.21.15.96:0/1625880399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1006f0 0x7f265c100b10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 -- 172.21.15.96:0/1625880399 >> 172.21.15.96:0/1625880399 conn(0x7f265c0fbc80 msgr2=0x7f265c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 -- 172.21.15.96:0/1625880399 shutdown_connections 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.391+0000 7f2661cea640 1 -- 172.21.15.96:0/1625880399 wait complete. 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 Processor -- start 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 -- start start 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 0x7f265c06a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1022a0 0x7f265c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f265c10c340 con 0x7f265c1018e0 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f265c10c1c0 con 0x7f265c1006f0 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f2661cea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f265c10c4c0 con 0x7f265c1022a0 2024-03-18T15:48:49.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41358/0 (socket says 172.21.15.96:41358) 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265affd640 1 -- 172.21.15.96:0/340396936 learned_addr learned my addr 172.21.15.96:0/340396936 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 0x7f265c06a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265bfff640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1022a0 0x7f265c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 -- 172.21.15.96:0/340396936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1022a0 msgr2=0x7f265c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1022a0 0x7f265c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 -- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 msgr2=0x7f265c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265b7fe640 1 -- 172.21.15.96:0/340396936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c19f750 con 0x7f265c1006f0 2024-03-18T15:48:49.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.392+0000 7f265affd640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f265b7fe640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 0x7f265c06a9a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f264c02c170 tx=0x7f264c02c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c041070 con 0x7f265c1006f0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c19f9e0 con 0x7f265c1006f0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f264c004450 con 0x7f265c1006f0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c19fea0 con 0x7f265c1006f0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.393+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c04b060 con 0x7f265c1006f0 2024-03-18T15:48:49.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.394+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c107b00 con 0x7f265c1006f0 2024-03-18T15:48:49.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.395+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f264c044070 con 0x7f265c1006f0 2024-03-18T15:48:49.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.395+0000 7f2658ff9640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 0x7f26440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.396+0000 7f265affd640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 0x7f26440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.396+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f264c0f0a50 con 0x7f265c1006f0 2024-03-18T15:48:49.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.396+0000 7f265affd640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 0x7f26440a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a710 tx=0x7f2650005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.399+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f264c0035f0 con 0x7f265c1006f0 2024-03-18T15:48:49.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.575+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f265c0623e0 con 0x7f265c1006f0 2024-03-18T15:48:49.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.575+0000 7f2658ff9640 1 -- 172.21.15.96:0/340396936 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f264c0bd470 con 0x7f265c1006f0 2024-03-18T15:48:49.576 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:48:49.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 msgr2=0x7f26440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 0x7f26440a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a710 tx=0x7f2650005f90 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 msgr2=0x7f265c06a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 0x7f265c06a9a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f264c02c170 tx=0x7f264c02c650 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 shutdown_connections 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f26440a2800 0x7f26440a4cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f265c1022a0 0x7f265c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f265c1018e0 0x7f265c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 --2- 172.21.15.96:0/340396936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f265c1006f0 0x7f265c06a9a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.577+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 >> 172.21.15.96:0/340396936 conn(0x7f265c0fbc80 msgr2=0x7f265c104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.578+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 shutdown_connections 2024-03-18T15:48:49.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.578+0000 7f2661cea640 1 -- 172.21.15.96:0/340396936 wait complete. 2024-03-18T15:48:49.590 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:48:49.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 -- 172.21.15.96:0/3911166111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9348104a70 msgr2=0x7f9348106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 --2- 172.21.15.96:0/3911166111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9348104a70 0x7f9348106e60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c02bc50 comp rx=0 tx=0).stop 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 -- 172.21.15.96:0/3911166111 shutdown_connections 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 --2- 172.21.15.96:0/3911166111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93481073a0 0x7f93481097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 --2- 172.21.15.96:0/3911166111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9348104a70 0x7f9348106e60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 --2- 172.21.15.96:0/3911166111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348102140 0x7f9348104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 -- 172.21.15.96:0/3911166111 >> 172.21.15.96:0/3911166111 conn(0x7f93480fbb50 msgr2=0x7f93480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 -- 172.21.15.96:0/3911166111 shutdown_connections 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.696+0000 7f934c99d640 1 -- 172.21.15.96:0/3911166111 wait complete. 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 Processor -- start 2024-03-18T15:48:49.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 -- start start 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 0x7f9348197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348104a70 0x7f93481980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f934810d430 con 0x7f93481073a0 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f934810d2b0 con 0x7f9348104a70 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f934c99d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f934810d5b0 con 0x7f9348102140 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.697+0000 7f9347fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f93477fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 0x7f9348197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41370/0 (socket says 172.21.15.96:41370) 2024-03-18T15:48:49.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9346ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348104a70 0x7f93481980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f93477fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 0x7f9348197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.96:42754/0 (socket says 172.21.15.96:42754) 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 -- 172.21.15.96:0/2991844645 learned_addr learned my addr 172.21.15.96:0/2991844645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 -- 172.21.15.96:0/2991844645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 msgr2=0x7f9348197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 0x7f9348197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 -- 172.21.15.96:0/2991844645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348104a70 msgr2=0x7f93481980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348104a70 0x7f93481980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 -- 172.21.15.96:0/2991844645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934819f7a0 con 0x7f93481073a0 2024-03-18T15:48:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.698+0000 7f9347fff640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f934000b970 tx=0x7f934000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.700+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340019070 con 0x7f93481073a0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.700+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934819fa90 con 0x7f93481073a0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.700+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f934000cb40 con 0x7f93481073a0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.700+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340007610 con 0x7f93481073a0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.700+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934819ffb0 con 0x7f93481073a0 2024-03-18T15:48:49.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.701+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93480fd640 con 0x7f93481073a0 2024-03-18T15:48:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.702+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f934001c070 con 0x7f93481073a0 2024-03-18T15:48:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.702+0000 7f9344ff9640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 0x7f93300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.702+0000 7f93477fe640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 0x7f93300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.703+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f93400cdfb0 con 0x7f93481073a0 2024-03-18T15:48:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.703+0000 7f93477fe640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 0x7f93300a4cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f93380090e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:49.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.707+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f934009a9d0 con 0x7f93481073a0 2024-03-18T15:48:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.881+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93480623e0 con 0x7f93481073a0 2024-03-18T15:48:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.881+0000 7f9344ff9640 1 -- 172.21.15.96:0/2991844645 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f934009ea70 con 0x7f93481073a0 2024-03-18T15:48:49.883 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 msgr2=0x7f93300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 0x7f93300a4cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9338006fd0 tx=0x7f93380090e0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 msgr2=0x7f934819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f934000b970 tx=0x7f934000be30 comp rx=0 tx=0).stop 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 shutdown_connections 2024-03-18T15:48:49.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f93300a2800 0x7f93300a4cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93481073a0 0x7f934819f130 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9348104a70 0x7f93481980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 --2- 172.21.15.96:0/2991844645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9348102140 0x7f9348197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 >> 172.21.15.96:0/2991844645 conn(0x7f93480fbb50 msgr2=0x7f93481029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 shutdown_connections 2024-03-18T15:48:49.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.883+0000 7f934c99d640 1 -- 172.21.15.96:0/2991844645 wait complete. 2024-03-18T15:48:49.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:49 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3663957697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:48:49.894 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:49 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/340396936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:48:49.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 -- 172.21.15.96:0/2895250906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 msgr2=0x7f0bb0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 --2- 172.21.15.96:0/2895250906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 0x7f0bb0100b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009900 tx=0x7f0ba002bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 -- 172.21.15.96:0/2895250906 shutdown_connections 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 --2- 172.21.15.96:0/2895250906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bb01022b0 0x7f0bb0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 --2- 172.21.15.96:0/2895250906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 --2- 172.21.15.96:0/2895250906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 0x7f0bb0100b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 -- 172.21.15.96:0/2895250906 >> 172.21.15.96:0/2895250906 conn(0x7f0bb00fbc90 msgr2=0x7f0bb00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:50.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 -- 172.21.15.96:0/2895250906 shutdown_connections 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:49.999+0000 7f0bb6790640 1 -- 172.21.15.96:0/2895250906 wait complete. 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 Processor -- start 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 -- start start 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 0x7f0bb0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bb01022b0 0x7f0bb019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010d500 con 0x7f0bb01018f0 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010d380 con 0x7f0bb0100700 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0bb6790640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010d680 con 0x7f0bb01022b0 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41394/0 (socket says 172.21.15.96:41394) 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 -- 172.21.15.96:0/755771050 learned_addr learned my addr 172.21.15.96:0/755771050 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 -- 172.21.15.96:0/755771050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bb01022b0 msgr2=0x7f0bb019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bb01022b0 0x7f0bb019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 -- 172.21.15.96:0/755771050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 msgr2=0x7f0bb0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 0x7f0bb0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.000+0000 7f0baf7fe640 1 -- 172.21.15.96:0/755771050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb019f760 con 0x7f0bb01018f0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0baf7fe640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9f0 tx=0x7f0ba400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400cd80 con 0x7f0bb01018f0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb019fa50 con 0x7f0bb01018f0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba4010910 con 0x7f0bb01018f0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40188f0 con 0x7f0bb01018f0 2024-03-18T15:48:50.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.001+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb019fff0 con 0x7f0bb01018f0 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.002+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb0107aa0 con 0x7f0bb01018f0 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.003+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0ba4010ac0 con 0x7f0bb01018f0 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.003+0000 7f0bad7fa640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 0x7f0b980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.004+0000 7f0baffff640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 0x7f0b980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.004+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0ba40cf3c0 con 0x7f0bb01018f0 2024-03-18T15:48:50.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.004+0000 7f0baffff640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 0x7f0b980a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0036040 tx=0x7f0ba0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:50.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.007+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0ba409c430 con 0x7f0bb01018f0 2024-03-18T15:48:50.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:49 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3663957697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:48:50.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:49 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/340396936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:48:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:49 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3663957697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:48:50.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:49 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/340396936' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:48:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.179+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0bb002d1b0 con 0x7f0bb01018f0 2024-03-18T15:48:50.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.180+0000 7f0bad7fa640 1 -- 172.21.15.96:0/755771050 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f0bb002d1b0 con 0x7f0bb01018f0 2024-03-18T15:48:50.181 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 msgr2=0x7f0b980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 0x7f0b980a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0036040 tx=0x7f0ba0000bc0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 msgr2=0x7f0bb0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e9f0 tx=0x7f0ba400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 shutdown_connections 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0b980a2800 0x7f0b980a4cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0bb01022b0 0x7f0bb019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0bb01018f0 0x7f0bb0198080 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 --2- 172.21.15.96:0/755771050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0bb0100700 0x7f0bb0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 >> 172.21.15.96:0/755771050 conn(0x7f0bb00fbc90 msgr2=0x7f0bb0104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:50.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 shutdown_connections 2024-03-18T15:48:50.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.182+0000 7f0bb6790640 1 -- 172.21.15.96:0/755771050 wait complete. 2024-03-18T15:48:50.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 -- 172.21.15.96:0/143846209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101ca0 msgr2=0x7fc9281020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 --2- 172.21.15.96:0/143846209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101ca0 0x7fc9281020c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc924009900 tx=0x7fc92402bc30 comp rx=0 tx=0).stop 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 -- 172.21.15.96:0/143846209 shutdown_connections 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 --2- 172.21.15.96:0/143846209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc928101390 0x7fc928108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 --2- 172.21.15.96:0/143846209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9281009d0 0x7fc928100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 --2- 172.21.15.96:0/143846209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101ca0 0x7fc9281020c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 -- 172.21.15.96:0/143846209 >> 172.21.15.96:0/143846209 conn(0x7fc9280fb400 msgr2=0x7fc9280fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 -- 172.21.15.96:0/143846209 shutdown_connections 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.294+0000 7fc92ff52640 1 -- 172.21.15.96:0/143846209 wait complete. 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 Processor -- start 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 -- start start 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9281009d0 0x7fc928197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc928101ca0 0x7fc92819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810d520 con 0x7fc928101390 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810d3a0 con 0x7fc928101ca0 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92ff52640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc92810d6a0 con 0x7fc9281009d0 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41408/0 (socket says 172.21.15.96:41408) 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 -- 172.21.15.96:0/2738277666 learned_addr learned my addr 172.21.15.96:0/2738277666 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:48:50.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92dcee640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9281009d0 0x7fc928197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92e4ef640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc928101ca0 0x7fc92819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 -- 172.21.15.96:0/2738277666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9281009d0 msgr2=0x7fc928197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9281009d0 0x7fc928197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 -- 172.21.15.96:0/2738277666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc928101ca0 msgr2=0x7fc92819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc928101ca0 0x7fc92819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 -- 172.21.15.96:0/2738277666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92819f7c0 con 0x7fc928101390 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.295+0000 7fc92d4ed640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc91800b4a0 tx=0x7fc91800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.296+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918014070 con 0x7fc928101390 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.296+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc918010af0 con 0x7fc928101390 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.296+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92819fab0 con 0x7fc928101390 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.296+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc918017d10 con 0x7fc928101390 2024-03-18T15:48:50.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.296+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281a0000 con 0x7fc928101390 2024-03-18T15:48:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.297+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc928107b10 con 0x7fc928101390 2024-03-18T15:48:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.298+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fc918017070 con 0x7fc928101390 2024-03-18T15:48:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.298+0000 7fc91effd640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 0x7fc9140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:48:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.298+0000 7fc92dcee640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 0x7fc9140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:48:50.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.298+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fc9180cdd10 con 0x7fc928101390 2024-03-18T15:48:50.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.299+0000 7fc92dcee640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 0x7fc9140a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc92402efd0 tx=0x7fc92402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:48:50.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.303+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fc91809a730 con 0x7fc928101390 2024-03-18T15:48:50.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.528+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fc9280623e0 con 0x7fc928101390 2024-03-18T15:48:50.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.536+0000 7fc91effd640 1 -- 172.21.15.96:0/2738277666 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+739829 (secure 0 0 0) 0x7fc91809e7d0 con 0x7fc928101390 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 20, 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout: "active_gid": 14152, 2024-03-18T15:48:50.556 INFO:teuthology.orchestra.run.smithi096.stdout: "active_name": "x", 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addrs": { 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3036369138 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3036369138 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addr": "172.21.15.96:6801/3036369138", 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "active_change": "2024-03-18T15:31:01.361112+0000", 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "active_mgr_features": 4540701547738038271, 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "available": true, 2024-03-18T15:48:50.558 INFO:teuthology.orchestra.run.smithi096.stdout: "standbys": [ 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24109, 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "z", 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.559 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T15:48:50.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.561 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.565 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-03-18T15:48:50.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.577 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-03-18T15:48:50.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-03-18T15:48:50.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.585 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-03-18T15:48:50.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-03-18T15:48:50.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T15:48:50.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-03-18T15:48:50.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-03-18T15:48:50.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.596 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.598 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.600 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-03-18T15:48:50.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-03-18T15:48:50.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-03-18T15:48:50.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.612 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-03-18T15:48:50.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.615 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-03-18T15:48:50.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-03-18T15:48:50.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.624 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-03-18T15:48:50.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.633 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-03-18T15:48:50.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-03-18T15:48:50.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-03-18T15:48:50.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-03-18T15:48:50.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T15:48:50.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.650 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-03-18T15:48:50.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-03-18T15:48:50.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-03-18T15:48:50.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:50.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.656 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.657 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.664 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-03-18T15:48:50.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-03-18T15:48:50.673 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-03-18T15:48:50.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-03-18T15:48:50.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-03-18T15:48:50.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-03-18T15:48:50.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.694 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.697 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T15:48:50.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-03-18T15:48:50.702 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T15:48:50.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T15:48:50.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T15:48:50.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.724 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T15:48:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.729 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T15:48:50.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T15:48:50.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.747 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T15:48:50.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T15:48:50.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.756 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.757 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T15:48:50.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.761 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.762 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.765 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-03-18T15:48:50.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.770 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.771 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.774 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.776 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-03-18T15:48:50.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:50.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.783 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.784 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:50.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:50.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.794 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-03-18T15:48:50.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.803 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.804 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:50.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.807 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.814 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.818 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:50.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T15:48:50.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.829 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.831 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-03-18T15:48:50.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:50.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.842 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.844 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.845 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.851 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.852 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.853 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.854 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-03-18T15:48:50.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.868 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.869 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.870 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.871 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.872 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.874 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.876 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-03-18T15:48:50.878 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.879 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.880 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.882 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.884 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.885 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.886 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.888 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-03-18T15:48:50.890 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.893 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.894 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.896 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.901 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.902 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.903 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.906 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-03-18T15:48:50.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.908 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.911 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.912 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.914 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-03-18T15:48:50.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:50.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.918 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T15:48:50.919 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.920 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.921 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.922 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.923 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-03-18T15:48:50.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T15:48:50.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.926 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-03-18T15:48:50.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-03-18T15:48:50.929 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.930 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.932 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.933 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.935 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.937 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.940 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.942 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.944 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.948 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.949 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.950 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.951 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.952 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.953 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.957 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.959 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.961 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.963 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.964 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.966 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.968 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-03-18T15:48:50.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-03-18T15:48:50.970 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-03-18T15:48:50.971 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-03-18T15:48:50.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.973 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.978 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.980 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.982 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.984 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.985 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.986 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.987 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.989 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.991 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.992 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.993 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.994 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:50.996 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.997 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:50.998 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:50.999 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.000 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.001 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.003 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.005 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-03-18T15:48:51.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-03-18T15:48:51.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-03-18T15:48:51.008 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-03-18T15:48:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.012 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.017 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.019 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.020 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.024 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.027 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.028 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.029 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.030 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.032 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.034 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.037 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.038 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-03-18T15:48:51.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.042 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.044 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.047 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24116, 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "y", 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-03-18T15:48:51.049 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.050 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.051 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.054 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.058 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.059 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.060 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-03-18T15:48:51.061 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-03-18T15:48:51.062 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T15:48:51.064 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:51.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-03-18T15:48:51.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-03-18T15:48:51.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.071 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.072 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.073 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.074 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.075 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.076 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.077 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-03-18T15:48:51.078 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.083 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.084 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.085 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.086 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.087 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T15:48:51.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-03-18T15:48:51.089 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.090 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.091 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T15:48:51.092 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.094 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.095 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-03-18T15:48:51.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-03-18T15:48:51.098 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-03-18T15:48:51.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.100 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T15:48:51.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.104 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.106 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T15:48:51.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.108 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.109 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.110 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-03-18T15:48:51.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-03-18T15:48:51.114 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-03-18T15:48:51.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.117 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.119 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.120 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.121 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-03-18T15:48:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T15:48:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-03-18T15:48:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-03-18T15:48:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T15:48:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T15:48:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T15:48:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T15:48:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T15:48:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T15:48:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-03-18T15:48:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T15:48:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-03-18T15:48:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T15:48:51.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T15:48:51.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.327 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.329 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-03-18T15:48:51.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T15:48:51.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-03-18T15:48:51.337 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.342 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-03-18T15:48:51.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.352 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.357 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.358 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.359 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.364 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.368 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-03-18T15:48:51.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.384 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.386 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.399 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.401 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.404 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-03-18T15:48:51.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-03-18T15:48:51.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-03-18T15:48:51.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.421 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.424 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-03-18T15:48:51.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-03-18T15:48:51.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.428 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.429 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.434 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.437 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-03-18T15:48:51.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.442 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.443 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.448 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "modules": [ 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard", 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "iostat", 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "nfs", 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: "restful" 2024-03-18T15:48:51.453 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.454 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.463 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-03-18T15:48:51.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-03-18T15:48:51.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-03-18T15:48:51.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-03-18T15:48:51.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:51.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:51.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.476 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.479 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-03-18T15:48:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.482 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.483 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-03-18T15:48:51.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.485 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-18T15:48:51.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-03-18T15:48:51.493 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.499 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-03-18T15:48:51.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-03-18T15:48:51.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-03-18T15:48:51.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-18T15:48:51.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-18T15:48:51.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.515 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-03-18T15:48:51.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-03-18T15:48:51.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-03-18T15:48:51.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.524 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.534 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-03-18T15:48:51.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-03-18T15:48:51.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.542 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-03-18T15:48:51.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-03-18T15:48:51.545 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.546 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.550 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.551 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.553 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-03-18T15:48:51.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.558 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.563 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.572 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-03-18T15:48:51.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-18T15:48:51.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.581 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.590 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-18T15:48:51.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-03-18T15:48:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-03-18T15:48:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-03-18T15:48:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-03-18T15:48:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-03-18T15:48:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-03-18T15:48:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-18T15:48:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-03-18T15:48:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-18T15:48:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-03-18T15:48:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-03-18T15:48:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-03-18T15:48:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-03-18T15:48:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-03-18T15:48:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-03-18T15:48:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-03-18T15:48:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-03-18T15:48:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-03-18T15:48:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-03-18T15:48:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.779 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.782 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.792 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.793 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.794 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-03-18T15:48:51.796 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.798 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.799 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.803 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.811 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.814 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-03-18T15:48:51.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-03-18T15:48:51.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.835 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.839 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.843 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.848 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-03-18T15:48:51.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.852 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-03-18T15:48:51.854 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-03-18T15:48:51.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-03-18T15:48:51.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-03-18T15:48:51.860 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-03-18T15:48:51.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "services": { 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard": "https://172.21.15.96:8443/" 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "always_on_modules": { 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "octopus": [ 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-03-18T15:48:51.865 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "pacific": [ 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-03-18T15:48:51.866 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "quincy": [ 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-03-18T15:48:51.867 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "reef": [ 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "squid": [ 2024-03-18T15:48:51.868 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "last_failure_osd_epoch": 3, 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "active_clients": [ 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.869 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 72482423 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "libcephsqlite", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-03-18T15:48:51.870 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3295206201 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1723493596 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3683302653 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-03-18T15:48:51.872 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:51.873 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:50 smithi193 ceph-mon[36702]: pgmap v615: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 13 op/s 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:50 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2991844645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:50 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/755771050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:50 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2738277666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:50 smithi137 ceph-mon[35868]: pgmap v615: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 13 op/s 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:50 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2991844645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:50 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/755771050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.874 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:50 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2738277666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 msgr2=0x7fc9140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 0x7fc9140a4cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc92402efd0 tx=0x7fc92402cf90 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 msgr2=0x7fc9281980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc91800b4a0 tx=0x7fc91800b960 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 shutdown_connections 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fc9140a2800 0x7fc9140a4cc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc928101ca0 0x7fc92819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc9281980e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 --2- 172.21.15.96:0/2738277666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9281009d0 0x7fc928197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.541+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 >> 172.21.15.96:0/2738277666 conn(0x7fc9280fb400 msgr2=0x7fc928104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:48:51.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.542+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 shutdown_connections 2024-03-18T15:48:51.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:48:50.542+0000 7fc92ff52640 1 -- 172.21.15.96:0/2738277666 wait complete. 2024-03-18T15:48:51.918 DEBUG:tasks.check_counter:Getting stats from a 2024-03-18T15:48:51.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-18T15:48:51.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:50 smithi096 ceph-mon[31930]: pgmap v615: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 13 op/s 2024-03-18T15:48:51.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:50 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2991844645' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:50 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/755771050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:48:51.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:50 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2738277666' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-18T15:48:52.169 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:48:52.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:52 smithi096 ceph-mon[31930]: pgmap v616: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:48:53.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:52 smithi137 ceph-mon[35868]: pgmap v616: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:48:53.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:52 smithi193 ceph-mon[36702]: pgmap v616: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1319, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1321, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 2425253, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 419443, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.109921378, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.035435179, 2024-03-18T15:48:54.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.028773504, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.035389322, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1319, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.059152913, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000044846 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.124 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 532, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 666, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 131438, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 486574, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.055465573, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.019896844, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.011756421, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.019291228, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 664, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.023456595, 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000035326 2024-03-18T15:48:54.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 317648 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 218, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 191, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 896172, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 111486, 2024-03-18T15:48:54.126 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.030244377, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011838260, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018289425, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.003434328, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 191, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009428297, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000049362 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 895904, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 111168 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3083, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.077516961, 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000025143 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.129 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 290, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 290, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 290, 2024-03-18T15:48:54.130 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-03-18T15:48:54.131 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-03-18T15:48:54.132 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-03-18T15:48:54.133 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-03-18T15:48:54.134 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 1515, 2024-03-18T15:48:54.135 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 3, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 6227102, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 6224810, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.136 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 1515 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 5, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 305, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 300, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 3, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 24, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 21, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 290, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 846, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 556, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-03-18T15:48:54.137 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 53508, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215420 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.142 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.145 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.146 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.149 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 2, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 1, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-03-18T15:48:54.150 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 2712, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:48:54.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:48:54.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:48:54.153 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 189463, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 52, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7728, 2024-03-18T15:48:54.154 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 163, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 219640, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 1200, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 1851, 2024-03-18T15:48:54.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 1851, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1851, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.487943352, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001344107 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1851, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1852 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 1851, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 1851, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-03-18T15:48:54.156 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1169, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 680, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-03-18T15:48:54.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-03-18T15:48:54.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 91, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 5, 2024-03-18T15:48:54.159 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 5, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-03-18T15:48:54.160 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:54.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2065, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 3253121, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2065, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2065, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3253121, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-03-18T15:48:54.162 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1851, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2032803, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1851, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2032803, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1851, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1851, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1851, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1851, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.164 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-03-18T15:48:54.165 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x5630142726e0": { 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:54.166 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:54.167 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:54.876 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-18T15:48:54.876 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-18T15:48:54.876 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-18T15:48:54.876 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-18T15:48:54.876 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-03-18T15:48:54.876 DEBUG:tasks.check_counter:Getting stats from d 2024-03-18T15:48:54.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-18T15:48:55.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:54 smithi137 ceph-mon[35868]: pgmap v617: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:48:55.131 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:48:55.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:54 smithi193 ceph-mon[36702]: pgmap v617: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:48:55.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:54 smithi096 ceph-mon[31930]: pgmap v617: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:48:56.900 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:56 smithi096 ceph-mon[31930]: pgmap v618: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 189, 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 184, 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 742192, 2024-03-18T15:48:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 112848, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.031703725, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011610484, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018703394, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.003552708, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 183, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.007724811, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000042212 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 742192, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 112848 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 140, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 160, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 102400, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.010850646, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.008391295, 2024-03-18T15:48:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001072967, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 138, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006973994, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000050536 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 102400 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:48:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3072, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000546974, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000033716, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000502588, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000660094, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000220031 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-03-18T15:48:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-03-18T15:48:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-03-18T15:48:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148967, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 29, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-03-18T15:48:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 190, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 715519, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 190, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 190, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 715519, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:56.910 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:48:57.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:56 smithi137 ceph-mon[35868]: pgmap v618: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:57.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:56 smithi193 ceph-mon[36702]: pgmap v618: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:48:57.597 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-18T15:48:57.598 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-18T15:48:57.598 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-18T15:48:57.598 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-18T15:48:57.598 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-03-18T15:48:57.598 DEBUG:tasks.check_counter:Getting stats from g 2024-03-18T15:48:57.598 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-18T15:48:57.841 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:48:58.880 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:48:58 smithi096 ceph-mon[31930]: pgmap v619: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:59.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:48:58 smithi137 ceph-mon[35868]: pgmap v619: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:59.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:48:58 smithi193 ceph-mon[36702]: pgmap v619: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 857, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 832, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1109088, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 316140, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.100166708, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.039994024, 2024-03-18T15:48:59.468 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.034045390, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.023444879, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 831, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.048792144, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000058714 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 893376, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 111984 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.469 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1303, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1437, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 8943195, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 734501, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.093302782, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.028106228, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.028659750, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.030778654, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1436, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.046800355, 2024-03-18T15:48:59.470 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000032590 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 321680 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 6, 2024-03-18T15:48:59.471 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1629, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1284, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001180224, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000063837, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000920908, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000086296, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002222832, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000555708 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.472 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-03-18T15:48:59.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3244, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.072648779, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000022394 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.474 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.475 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 1117, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 1117, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 1117, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-03-18T15:48:59.476 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-03-18T15:48:59.477 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-03-18T15:48:59.478 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-03-18T15:48:59.479 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:48:59.480 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 5926, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-03-18T15:48:59.481 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 8, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 12802651, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 12801513, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.482 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 5926 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 10, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 1138, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 1128, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 8, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 51, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 43, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 1117, 2024-03-18T15:48:59.483 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 2402, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 1285, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 53536, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215420 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-03-18T15:48:59.484 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.485 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.489 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-03-18T15:48:59.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.489 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.489 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.490 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.491 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.492 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.494 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.496 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 1, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 0, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 1626, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.497 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:48:59.498 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:48:59.499 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 197463, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 54, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-03-18T15:48:59.500 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 7728, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 163, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 819288, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 4563, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-03-18T15:48:59.501 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 1944, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 1944, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1944, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 9.281788142, 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004774582 2024-03-18T15:48:59.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1944, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1945 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 1944, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 1944, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1161, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 781, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-03-18T15:48:59.503 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-03-18T15:48:59.505 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-03-18T15:48:59.506 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 91, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 5, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 6, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 6, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-03-18T15:48:59.507 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T15:48:59.508 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2160, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 9845516, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2160, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:59.509 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2160, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 9845516, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:59.510 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1944, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 8609572, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1944, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 8609572, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:59.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1944, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1944, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1944, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1944, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:59.513 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x556a040826e0": { 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:48:59.514 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:48:59.515 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:49:00.241 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-18T15:49:00.241 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-18T15:49:00.241 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-18T15:49:00.241 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-18T15:49:00.241 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-03-18T15:49:00.241 DEBUG:tasks.check_counter:Getting stats from j 2024-03-18T15:49:00.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-18T15:49:00.481 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.a/config 2024-03-18T15:49:00.884 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:00 smithi096 ceph-mon[31930]: pgmap v620: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:01.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:00 smithi137 ceph-mon[35868]: pgmap v620: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:01.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:00 smithi193 ceph-mon[36702]: pgmap v620: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001170533, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000310691, 2024-03-18T15:49:02.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:49:02.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 141, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 160, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 102576, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.010714863, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.007962500, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001432394, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 139, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006694749, 2024-03-18T15:49:02.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000048163 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 102576 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 195, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 188, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 745808, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 114576, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-03-18T15:49:02.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.029958628, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011192262, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018522166, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.003657831, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 188, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009935052, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000052846 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:49:02.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 745808, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 114576 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:02.326 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:02.327 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 149007, 2024-03-18T15:49:02.328 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 30, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-03-18T15:49:02.329 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 192, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 715779, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 192, 2024-03-18T15:49:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 192, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 715779, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-03-18T15:49:02.331 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-03-18T15:49:02.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:02 smithi096 ceph-mon[31930]: pgmap v621: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:02.969 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:02 smithi193 ceph-mon[36702]: pgmap v621: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:02.989 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-18T15:49:02.989 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-18T15:49:02.990 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-18T15:49:02.990 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-18T15:49:02.990 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-03-18T15:49:02.990 DEBUG:tasks.check_counter:Getting stats from b 2024-03-18T15:49:02.990 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-18T15:49:03.015 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:02 smithi137 ceph-mon[35868]: pgmap v621: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:03.280 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:49:04.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:03 smithi137 ceph-mon[35868]: pgmap v622: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:04.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:03 smithi193 ceph-mon[36702]: pgmap v622: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:04.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:03 smithi096 ceph-mon[31930]: pgmap v622: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:05.164 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3034, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3104, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1808832, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 9083438, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.213166813, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.083073465, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.099313490, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.038078766, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3103, 2024-03-18T15:49:05.165 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.085542963, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000027567 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2889, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000442812, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000153 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 920096, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 125808 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 27250, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 31746, 2024-03-18T15:49:05.166 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 8246944, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 17692981, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 15, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 7, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.938735826, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.457813941, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 1.001114909, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.028482869, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 31744, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.220388064, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000038444 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 26701, 2024-03-18T15:49:05.167 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005174428, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000193 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 326160 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 13589, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 11492, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 6065012, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 3551811, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 24, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 9, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.679243914, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.263036930, 2024-03-18T15:49:05.168 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.485353368, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.031048582, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11491, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.229771179, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000019995 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 22097, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002726669, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000123 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.169 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2304, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.688808032, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001167017 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.170 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2203, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.144737805, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000065700 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 15641, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 14565, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 14565, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 251.858111288, 2024-03-18T15:49:05.171 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.017292009 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 1076, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 38, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 3, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 25, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 6, 2024-03-18T15:49:05.172 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 9, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 10, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 13, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 356, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 4, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 33098, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 27383, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 1076, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 184, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 294, 2024-03-18T15:49:05.173 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 25, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 12, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 4, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 541, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 2337, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 2458, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 40839530, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 2337, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:05.174 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 2337, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 23, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 990, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 2065, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 2143, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 1047, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 143, 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 4057 2024-03-18T15:49:05.175 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 528, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 2851, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 1647, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 1891, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 301, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 1047, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 1047, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:05.176 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 25, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 3, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 5525, 2024-03-18T15:49:05.177 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 5525, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 7, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 7, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 12260239, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-03-18T15:49:05.178 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 295, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.644172896, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008963297 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 30, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1253, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1223, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 29, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 1461, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1432, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 25, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 2668, 2024-03-18T15:49:05.179 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 2643, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 1480, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1480, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 65464, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223612 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 18462, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 26, 2024-03-18T15:49:05.180 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 15641, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 8, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 2269, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 793, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 7.005689027, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008834412 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 347, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3.466079914, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.009988702 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.181 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 11279, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 231.348060238, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.020511398 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:05.182 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:05.183 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.185 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-18T15:49:05.186 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 728, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.937896971, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002661946 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 25, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.027907580, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001116303 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 795, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6.603820979, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008306693 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:05.187 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:05.188 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:05.189 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.190 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 598, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.468656579, 2024-03-18T15:49:05.191 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002455947 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-18T15:49:05.192 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 1304, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 1304, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 483, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:05.193 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:05.194 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:05.195 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 300322, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 132, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 616, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 7, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:05.196 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 122301, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 484, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 4126, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 8867843, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 29, 2024-03-18T15:49:05.197 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 4126, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4126, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 17.086696004, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.004141225 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4126, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4345 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 4126, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2420, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 1706, 2024-03-18T15:49:05.198 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 83, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 378, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 81, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 3, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 1060, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2337, 2024-03-18T15:49:05.199 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-18T15:49:05.200 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 91, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-18T15:49:05.201 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 5, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 311, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 62 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:05.202 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 1047, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 1047, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.203 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 43869, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 12565792, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 43869, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 43869, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 12565792, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.204 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 4126, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 8973590, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 4126, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 8973590, 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:05.205 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 4126, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 4126, 2024-03-18T15:49:05.206 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 4126, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 4126, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1047, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 105747, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1047, 2024-03-18T15:49:05.207 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 83, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 105747, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x564121154500": { 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5525, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 8065935, 2024-03-18T15:49:05.208 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5525, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 295, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 8065935, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:05.209 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-18T15:49:05.884 INFO:tasks.check_counter:Daemon mds.b mds.exported=25 2024-03-18T15:49:05.884 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-03-18T15:49:05.885 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=528 2024-03-18T15:49:05.885 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=2851 2024-03-18T15:49:05.885 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=3 2024-03-18T15:49:05.885 DEBUG:tasks.check_counter:Getting stats from e 2024-03-18T15:49:05.885 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-18T15:49:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:49:06.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:06 smithi137 ceph-mon[35868]: pgmap v623: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:06.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:06 smithi096 ceph-mon[31930]: pgmap v623: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:07.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:06 smithi193 ceph-mon[36702]: pgmap v623: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:07.966 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 192, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 187, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 241744, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 69532, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.017822095, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.005723065, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.006950853, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.004255351, 2024-03-18T15:49:07.967 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 186, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006118177, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000032893 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 194208, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 25008 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 143, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 173, 2024-03-18T15:49:07.968 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 39396, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 182966, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012171365, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003516335, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.003867253, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.003777015, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 171, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005352584, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000031301 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.969 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 137744 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 181, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 181, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2523647, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 56741, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012527247, 2024-03-18T15:49:07.970 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.003002537, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.005526960, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.003121928, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 180, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.007720742, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000042893 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T15:49:07.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 749, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.017763788, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000023716 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:07.972 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-18T15:49:07.973 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-18T15:49:07.974 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 673, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 462, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 211, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 12, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:07.975 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:07.976 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-03-18T15:49:07.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:07.978 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-18T15:49:07.979 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 1446, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 2, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 8, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 11310578, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 11338359, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 11337221, 2024-03-18T15:49:07.980 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 1446 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 12, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 687, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 675, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 11, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 48, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 37, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 673, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 1348, 2024-03-18T15:49:07.981 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 675, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 47896, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215396 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.982 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.983 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.984 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.986 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.987 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.991 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-18T15:49:07.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:07.993 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 0, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1846, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-18T15:49:07.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:07.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 189507, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 47, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:07.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 515325, 2024-03-18T15:49:07.998 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2922, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-18T15:49:07.999 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 459, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 26, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 4, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 459, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 459, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.203996389, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000444436 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-18T15:49:08.000 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 459, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 460 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 459, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 459, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 278, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 178, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-03-18T15:49:08.001 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 2, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:08.002 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-18T15:49:08.003 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-18T15:49:08.004 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 5, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 5, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.005 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-03-18T15:49:08.006 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 512, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2752996, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 512, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 512, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2752996, 2024-03-18T15:49:08.007 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 459, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 2474691, 2024-03-18T15:49:08.008 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 459, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2474691, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:08.009 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 459, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 459, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 459, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 459, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:08.010 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55b5725126e0": { 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:08.011 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.012 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:08.013 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-18T15:49:08.778 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-18T15:49:08.778 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-18T15:49:08.778 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-18T15:49:08.778 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-03-18T15:49:08.778 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-03-18T15:49:08.779 DEBUG:tasks.check_counter:Getting stats from h 2024-03-18T15:49:08.779 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-18T15:49:08.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:08 smithi137 ceph-mon[35868]: pgmap v624: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:08.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:08 smithi096 ceph-mon[31930]: pgmap v624: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:09.085 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:49:09.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:08 smithi193 ceph-mon[36702]: pgmap v624: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:10.847 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:10 smithi137 ceph-mon[35868]: pgmap v625: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 28303, 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 32895, 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 8760389, 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 22896432, 2024-03-18T15:49:10.848 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 17, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 1.014742510, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.499674056, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.766375631, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.041295183, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 32889, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.854728816, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000025988 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 27721, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.005799120, 2024-03-18T15:49:10.849 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000209 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3040, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 3207, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1735664, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 1343333, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.186323903, 2024-03-18T15:49:10.850 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.065518944, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.092113063, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.033106139, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3211, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.088541887, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000027574 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2676, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000392442, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000146 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 909632, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 158000 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.851 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 14659, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 12741, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 6199331, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 8097938, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 28, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 10, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.727779259, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.287095694, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.508719753, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.036780651, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 12739, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.258282805, 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000020274 2024-03-18T15:49:10.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 23615, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002860030, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000121 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 326784 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:10.853 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3041, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 3.025678515, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000994961 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2270, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.153758890, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000067735 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.854 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 15233, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 15153, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 15153, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 328.870851734, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.021703349 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 80, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 38, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 3, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-18T15:49:10.855 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 25, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 6, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 9, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 10, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 13, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 352, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 1, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 1, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 4, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 33541, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 28874, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 80, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 176, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:10.856 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 204, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 24, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 11, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 3, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 598, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 3309, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 2458, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 40839530, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 3309, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:10.857 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 3309, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 20, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 920, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 2079, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 2032, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 1110, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 56, 2024-03-18T15:49:10.858 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 4266 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 575, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 2952, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 1781, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 1954, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 489, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 1110, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 1110, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-18T15:49:10.859 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 24, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 3, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-18T15:49:10.860 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 5926, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 5926, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 8, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 8, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 2, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4194304, 2024-03-18T15:49:10.861 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 12802651, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 286, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2.615550231, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.009145280 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 30, 2024-03-18T15:49:10.862 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 1313, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 1283, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 29, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 1380, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1351, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 25, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 2769, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 2744, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 1, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 1585, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 1584, 2024-03-18T15:49:10.863 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 67424, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 223612 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 18015, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 143, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 15233, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 8, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 2612, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 837, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6.763183991, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008080267 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-18T15:49:10.864 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 12151, 2024-03-18T15:49:10.865 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 313.388337691, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.025791156 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000084522, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000084522 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.866 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.867 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.868 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 682, 2024-03-18T15:49:10.869 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.702089746, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001029457 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 24, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.028292130, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001178838 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 837, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 6.856601413, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.008191877 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.870 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.872 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 621, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 1.132262241, 2024-03-18T15:49:10.874 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001823288 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-18T15:49:10.875 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 1183, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 1183, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 532, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:10.876 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:10.877 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 302299, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 140, 2024-03-18T15:49:10.878 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 880, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 10, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 122869, 2024-03-18T15:49:10.879 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 499, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 5160, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 9404000, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 33, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 5160, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5160, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 13.286762294, 2024-03-18T15:49:10.880 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002574953 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 5160, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 5403 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 5160, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 3386, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 1774, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 0, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 10, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 77, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 363, 2024-03-18T15:49:10.881 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 76, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 3, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 1124, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 3309, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 20, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-18T15:49:10.882 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-18T15:49:10.883 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 91, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 5, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 355, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-03-18T15:49:10.884 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 66 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-18T15:49:10.885 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 6, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 1110, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 1110, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 45998, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 12952270, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 45998, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:10.886 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 45998, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 12952270, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:10.887 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 5160, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 9516110, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5160, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 9516110, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:10.888 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 5160, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 5160, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 5160, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 5160, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 1110, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 112110, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 1110, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 77, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 112110, 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:10.890 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x56475dd1a500": { 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 5926, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 8608347, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 5926, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:10.891 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 286, 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 8608347, 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:10.892 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-18T15:49:10.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:10 smithi096 ceph-mon[31930]: pgmap v625: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:11.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:10 smithi193 ceph-mon[36702]: pgmap v625: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:11.643 INFO:tasks.check_counter:Daemon mds.h mds.exported=24 2024-03-18T15:49:11.643 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-03-18T15:49:11.643 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=575 2024-03-18T15:49:11.643 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2952 2024-03-18T15:49:11.643 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=3 2024-03-18T15:49:11.643 DEBUG:tasks.check_counter:Getting stats from k 2024-03-18T15:49:11.644 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-18T15:49:11.946 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.b/config 2024-03-18T15:49:12.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:12 smithi096 ceph-mon[31930]: pgmap v626: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:13.104 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:12 smithi137 ceph-mon[35868]: pgmap v626: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:13.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:12 smithi193 ceph-mon[36702]: pgmap v626: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:13.705 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-18T15:49:13.705 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:13.705 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 907, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 907, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 193196006, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 287666, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 8, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 4, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.138672907, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.016563364, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.099254491, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.017336221, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 906, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.021742113, 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000023997 2024-03-18T15:49:13.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1092, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1067, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 207505718, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 394111, 2024-03-18T15:49:13.707 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.179362042, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.021599775, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.137307585, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.020026617, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1066, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.026641609, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000024992 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 896096, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 118400 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1063, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 1206, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 126160687, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 663412, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.134857963, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.024844402, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.079394908, 2024-03-18T15:49:13.709 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.023078630, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1204, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.134956909, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000112090 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 325696 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.710 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4461, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.221657700, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000049687 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-03-18T15:49:13.711 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 0, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 0, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-03-18T15:49:13.712 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 43479, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 10490, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 32989, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 28493, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 1, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 0, 2024-03-18T15:49:13.713 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:13.714 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 0, 2024-03-18T15:49:13.715 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 0 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-03-18T15:49:13.716 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-03-18T15:49:13.717 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 0, 2024-03-18T15:49:13.718 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 234267, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 131, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 21, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 0, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 333812774, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 529328370, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 529326383, 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-03-18T15:49:13.719 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 234267 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 17, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 43841, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 43824, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 25, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 1401, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 1376, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 43479, 2024-03-18T15:49:13.720 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 94560, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 51081, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 123948, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 215420 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-03-18T15:49:13.721 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.722 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.723 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.724 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-03-18T15:49:13.725 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.725 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.725 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.725 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:13.726 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.726 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.727 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.728 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.729 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:13.730 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.731 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.732 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.733 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.734 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 1, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 1, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 1, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 0, 2024-03-18T15:49:13.735 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 1851, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:13.736 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:13.737 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:13.738 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 240650, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 84, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:13.739 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 163, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 30680428, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 174459, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.740 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 2832, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 21, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 3, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 2832, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2832, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 4.977910651, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001757736 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2832, 2024-03-18T15:49:13.741 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 2833 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 2832, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 2832, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 1668, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 1162, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-03-18T15:49:13.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 1, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-03-18T15:49:13.743 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-03-18T15:49:13.744 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 91, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 5, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 11, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 11, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 2, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 0 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:13.745 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.746 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 3058, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 526572675, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 3058, 2024-03-18T15:49:13.747 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 3058, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 526572675, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:13.748 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2832, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 525184507, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2832, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 525184507, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:13.749 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 2832, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 2832, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2832, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2832, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.751 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55d93b3506e0": { 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-03-18T15:49:13.752 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-18T15:49:13.753 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-18T15:49:14.526 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-18T15:49:14.527 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-18T15:49:14.527 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-18T15:49:14.529 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-18T15:49:14.529 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-03-18T15:49:14.529 DEBUG:tasks.check_counter:Getting stats from c 2024-03-18T15:49:14.530 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-18T15:49:14.823 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:49:14.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:14 smithi137 ceph-mon[35868]: pgmap v627: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:14.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:14 smithi193 ceph-mon[36702]: pgmap v627: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:14.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:14 smithi096 ceph-mon[31930]: pgmap v627: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:16.799 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 333951, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 516686, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 100922401, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 522749254, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 15, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 17.631239047, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 9.035033864, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 385.656460411, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 1.278170530, 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:16.800 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 516684, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 30.990645193, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000059979 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 438989, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.088616687, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000201 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 82853, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 84926, 2024-03-18T15:49:16.801 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 26584479, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 233170744, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 8.181694571, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 4.170365163, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 1.775854778, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 1.885236336, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 84926, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 23.148048876, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000272567 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 22098, 2024-03-18T15:49:16.802 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.004670591, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000211 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 923184, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 132704 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 63073, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 70658, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 22709461, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 109890274, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 28, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 10, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 5.246272301, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 2.169423050, 2024-03-18T15:49:16.803 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 1.727894531, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 1.184855609, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 70656, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 9.276926226, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000131297 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 41279, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.008600132, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000208 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 330400 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-03-18T15:49:16.804 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 35671, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 350.796750861, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009834228 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 72861, 2024-03-18T15:49:16.805 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 45.642619884, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000626434 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 174917, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 171939, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 171939, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6600.597327684, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.038389180 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 2978, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 1, 2024-03-18T15:49:16.806 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 1885, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 140, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 18, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 899, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 551, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 241, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 107, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 111, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 2734, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 6, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 272533, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 256622, 2024-03-18T15:49:16.807 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 2966, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 10, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 1668, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 13, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 1699, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 83, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 310, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 93924, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 2458, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 40839530, 2024-03-18T15:49:16.808 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 93924, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 666, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 666, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 93258, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 24, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 29244, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 92506, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 5, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:16.809 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 93041, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 44267, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 5660, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 91017 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 3976, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 14, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 9, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 16, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 7878, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 4, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 43033, 2024-03-18T15:49:16.810 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 43029, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 13, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 158, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:16.811 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 234267, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 137486, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 135191, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 99076, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 2295, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 302, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 174, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 171, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 131, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 9, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-03-18T15:49:16.812 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 3, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 333812774, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 529328370, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3213, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 763.379925192, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.237591013 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 37, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 43864, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 43827, 2024-03-18T15:49:16.813 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 46, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1422, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1376, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 899, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 97150, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 96251, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 44687, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 44687, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 150448, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223612 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 182297, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 6934, 2024-03-18T15:49:16.814 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 174917, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 20, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 14132, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 43609, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1010.725797463, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.023177000 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6412, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1854.051725169, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.289153419 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.815 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 45275, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2290.454236993, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.050589823 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.816 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.817 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.818 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 159, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 5.954467053, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.037449478 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.819 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6840, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 157.429701152, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.023016038 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1104, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.679745158, 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003333102 2024-03-18T15:49:16.820 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 43337, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1189.485024253, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.027447331 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.821 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 163, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 34.016309523, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.208689015 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.822 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.042469221, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.008493844 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.823 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-03-18T15:49:16.824 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002084410, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001042205 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 25033, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 54.755767289, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002187343 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.825 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 1, 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-03-18T15:49:16.826 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 34195, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 34195, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 551, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:16.827 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:16.828 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:16.829 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 316167, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 138, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:16.830 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 828173, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 10401, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 159128, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 573074817, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 159128, 2024-03-18T15:49:16.831 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 159128, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 696.781949838, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004378751 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 159128, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 187547 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 159128, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 95088, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 64040, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 594, 2024-03-18T15:49:16.832 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 12198, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1163, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 4376, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 527, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 140, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 44441, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 93925, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 24396, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:16.833 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 5, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-03-18T15:49:16.834 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 91, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 5, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-03-18T15:49:16.835 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 4634, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 1146 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 14, 2024-03-18T15:49:16.836 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 44122, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 43029, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 479873, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 111526821, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 479873, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:16.837 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 479873, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 111526821, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:16.838 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 159128, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 577421834, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 159128, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 577421834, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:16.839 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 159128, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 159128, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 159128, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 159128, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:16.840 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 43029, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 4347033, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 43029, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1163, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 4347033, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:16.841 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x559b0c9b2500": { 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 234267, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 525134066, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 234267, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3213, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 525134066, 2024-03-18T15:49:16.842 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:16.843 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-18T15:49:16.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:16 smithi193 ceph-mon[36702]: pgmap v628: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:16.936 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:16 smithi096 ceph-mon[31930]: pgmap v628: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:17.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:16 smithi137 ceph-mon[35868]: pgmap v628: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:17.638 INFO:tasks.check_counter:Daemon mds.c mds.exported=13 2024-03-18T15:49:17.638 INFO:tasks.check_counter:Daemon mds.c mds.imported=83 2024-03-18T15:49:17.638 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=3976 2024-03-18T15:49:17.639 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-03-18T15:49:17.639 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=140 2024-03-18T15:49:17.639 DEBUG:tasks.check_counter:Getting stats from f 2024-03-18T15:49:17.639 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-18T15:49:17.934 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:49:19.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:18 smithi137 ceph-mon[35868]: pgmap v629: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:19.130 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:18 smithi193 ceph-mon[36702]: pgmap v629: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:19.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:18 smithi096 ceph-mon[31930]: pgmap v629: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:20.140 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-18T15:49:20.140 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:20.140 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1069, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 869, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 432234, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 2528009, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 18, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.077234460, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.029120828, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.041083165, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.009371081, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 868, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.043757888, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000050412 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.141 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1249, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000204958, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000164 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 5584, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 3920 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 6685, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 8204, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2012622, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 4331889, 2024-03-18T15:49:20.142 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 15, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.328235937, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.156524318, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.261287789, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.008590063, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8202, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.543580128, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000066274 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6997, 2024-03-18T15:49:20.143 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002506998, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000358 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 329360 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 12346, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 8516, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 6302644, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 2294835, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 21, 2024-03-18T15:49:20.144 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.786712170, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.344608389, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.569173318, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.010785269, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8515, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.242220445, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000028446 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 19651, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003623336, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000184 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.145 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 911504, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 124400 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 502, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.156746008, 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002304274 2024-03-18T15:49:20.146 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 512, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.077356680, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000151087 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 4314, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 3052, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3052, 2024-03-18T15:49:20.147 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 44.081137423, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.014443360 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 1262, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 20, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 1, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 25, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 6, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 9, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2024-03-18T15:49:20.148 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 13, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 372, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 1, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 1, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 4, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 9898, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 7332, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 1260, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 196, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 77, 2024-03-18T15:49:20.149 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 8, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 19, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 1, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 44, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 452, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 2458, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 40839530, 2024-03-18T15:49:20.150 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 452, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 452, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 2, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 229, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 556, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:20.151 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 496, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 268, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 7, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 1367 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 116, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 3560, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 2142, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 2453, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 91, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 288, 2024-03-18T15:49:20.152 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 288, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 8, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 1, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:20.153 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 1515, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1515, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 3, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 3, 2024-03-18T15:49:20.154 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 6227102, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 72, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.585628852, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.022022622 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 30, 2024-03-18T15:49:20.155 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 504, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 474, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 29, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1590, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1561, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 25, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 1213, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1188, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 1, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 293, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 292, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 60392, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 223612 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-03-18T15:49:20.156 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 5318, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 19, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 4314, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 12, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 857, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 266, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.767022490, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.010402340 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001671299, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001671299 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.157 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1940, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 37.644604384, 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.019404435 2024-03-18T15:49:20.158 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000133845, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000133845 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.159 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-03-18T15:49:20.160 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-03-18T15:49:20.161 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 297, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.313039229, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001054004 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-03-18T15:49:20.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.015076127, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001884515 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 267, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.433925459, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009115825 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:20.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:20.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:20.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.166 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 272, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.905664590, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003329649 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.167 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 1158, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 1158, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 541, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-03-18T15:49:20.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:20.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:20.170 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 291295, 2024-03-18T15:49:20.171 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 121, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 528, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 6, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:20.172 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 122509, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 478, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-03-18T15:49:20.173 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 947, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 2315344, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 947, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 947, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.263948230, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003446619 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.174 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 947, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1016 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 947, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 468, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 479, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 16, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 88, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 27, 2024-03-18T15:49:20.175 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 1, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 300, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 452, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-03-18T15:49:20.176 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-03-18T15:49:20.177 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 91, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 5, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 90, 2024-03-18T15:49:20.178 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 12 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.179 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 288, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 288, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 20096, 2024-03-18T15:49:20.180 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 7022683, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 20096, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 20096, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 7022683, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:20.181 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 947, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2344432, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 947, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2344432, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.182 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 947, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 947, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 947, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 947, 2024-03-18T15:49:20.183 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 288, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 29088, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 288, 2024-03-18T15:49:20.184 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 16, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 29088, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5568497b0500": { 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1515, 2024-03-18T15:49:20.185 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2032798, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1515, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 72, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2032798, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:20.186 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-18T15:49:20.880 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:20 smithi193 ceph-mon[36702]: pgmap v630: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:20.912 INFO:tasks.check_counter:Daemon mds.f mds.exported=8 2024-03-18T15:49:20.912 INFO:tasks.check_counter:Daemon mds.f mds.imported=1 2024-03-18T15:49:20.912 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=116 2024-03-18T15:49:20.912 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=3560 2024-03-18T15:49:20.912 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=1 2024-03-18T15:49:20.912 DEBUG:tasks.check_counter:Getting stats from i 2024-03-18T15:49:20.913 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-18T15:49:21.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:20 smithi137 ceph-mon[35868]: pgmap v630: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:21.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:20 smithi096 ceph-mon[31930]: pgmap v630: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:21.284 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:49:22.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:22 smithi193 ceph-mon[36702]: pgmap v631: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:22.890 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:22 smithi193 ceph-mon[36702]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:49:22.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:22 smithi096 ceph-mon[31930]: pgmap v631: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:22.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:22 smithi096 ceph-mon[31930]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:49:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:22 smithi137 ceph-mon[35868]: pgmap v631: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-03-18T15:49:23.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:22 smithi137 ceph-mon[35868]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-18T15:49:23.350 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 19067, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 21633, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 6663321, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 17076372, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 45, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.960482423, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.443802898, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.740981930, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.068507295, 2024-03-18T15:49:23.351 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 21633, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.663001983, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000076873 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 19576, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.004857190, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000248 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 990000, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 170496 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:23.352 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 15649, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 12142, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 7151755, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3996407, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 1.049470570, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.485086748, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.637736519, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.055420072, 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:23.353 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 12144, 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.311973522, 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000272725 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 23162, 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.004206553, 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000181 2024-03-18T15:49:23.354 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 335840 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4212, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 3921, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1669517, 2024-03-18T15:49:23.355 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 8152255, 2024-03-18T15:49:23.356 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 150, 2024-03-18T15:49:23.356 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 8, 2024-03-18T15:49:23.356 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.356560313, 2024-03-18T15:49:23.356 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.126353913, 2024-03-18T15:49:23.356 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.152857712, 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.069085203, 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3915, 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.450700778, 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000625977 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:23.357 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4171, 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000761433, 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000182 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 41632, 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 1456 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.358 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:23.359 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2367, 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.018684914, 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001275321 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:23.360 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:23.361 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1439, 2024-03-18T15:49:23.361 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.140992869, 2024-03-18T15:49:23.361 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000097979 2024-03-18T15:49:23.361 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.361 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 12940, 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 11902, 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 11902, 2024-03-18T15:49:23.362 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 254.383244467, 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.021373151 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 1038, 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 14, 2024-03-18T15:49:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 5, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 424, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 3, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 33, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 11, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 12, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2024-03-18T15:49:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 17, 2024-03-18T15:49:23.365 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 871, 2024-03-18T15:49:23.365 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 1, 2024-03-18T15:49:23.365 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 1, 2024-03-18T15:49:23.365 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 3, 2024-03-18T15:49:23.365 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 27512, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 21456, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 1038, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 219, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 122, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 26, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 268, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 5, 2024-03-18T15:49:23.366 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 516, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 1749, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 1749, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 97, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 11, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 86, 2024-03-18T15:49:23.367 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 1652, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 31, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 533, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 1339, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 1395, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 820, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 40, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 2797 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 209, 2024-03-18T15:49:23.368 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 3912, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 2306, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 2602, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 161, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 1, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 747, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 746, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:23.369 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 11, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 26, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 3, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.370 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 4217, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 4206, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 4206, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 11, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 3, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 39, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 38, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 38, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 1, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 1, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-03-18T15:49:23.371 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 1, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 11310578, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 11338359, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 298, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.768227831, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.005933650 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 35, 2024-03-18T15:49:23.372 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 1166, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 1131, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 30, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 1815, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 1785, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 33, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 2808, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 2775, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 1, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 1486, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 1485, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 70884, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 231804 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-03-18T15:49:23.373 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 14792, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 30, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 12940, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 19, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 1640, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 554, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 5.248118115, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009473137 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-03-18T15:49:23.374 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 108, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.495359843, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004586665 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.375 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9554, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 237.636492696, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.024872984 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.376 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.377 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.378 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-03-18T15:49:23.378 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.378 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.378 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.378 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 15, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017766623, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001184441 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.379 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 449, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.048248556, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004561800 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 38, 2024-03-18T15:49:23.380 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.188390840, 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004957653 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 444, 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6.669756376, 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.015021973 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:23.381 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.382 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5, 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.057773849, 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011554769 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.383 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.384 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000799475, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000799475 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.385 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.386 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.018381682, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002297710 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-03-18T15:49:23.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001035347, 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001035347 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 725, 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.001121065, 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002760166 2024-03-18T15:49:23.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.389 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-03-18T15:49:23.390 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 2046, 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 2046, 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 556, 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:23.391 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:23.392 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:23.393 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:23.394 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 297650, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 119, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 528, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 6, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-03-18T15:49:23.395 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-03-18T15:49:23.396 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 144514, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 892, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-03-18T15:49:23.397 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 5160, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 9399437, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 37, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 5160, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5160, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 55.280114015, 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.010713200 2024-03-18T15:49:23.398 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5160, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 8002 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 5160, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1846, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 3314, 2024-03-18T15:49:23.399 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 377, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 951, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 62, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 349, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 174, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-03-18T15:49:23.400 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 3, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 1260, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1779, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 1902, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:23.401 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-03-18T15:49:23.402 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-03-18T15:49:23.403 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 91, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 6, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 1006, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 43, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 96 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.404 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 6, 2024-03-18T15:49:23.405 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 746, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 746, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 38924, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 12219229, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 38924, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:23.406 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 38924, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12219229, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:23.407 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5160, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 9474365, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5160, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 9474365, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.408 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 5160, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 5160, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 5160, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5160, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 746, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 75346, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 746, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:23.410 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 51, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 75346, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x563e47f34500": { 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 4217, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 7144055, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:23.411 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 4217, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 298, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 7144055, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:23.412 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-18T15:49:24.221 INFO:tasks.check_counter:Daemon mds.i mds.exported=26 2024-03-18T15:49:24.222 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-03-18T15:49:24.222 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=209 2024-03-18T15:49:24.222 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=3912 2024-03-18T15:49:24.222 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=3 2024-03-18T15:49:24.222 DEBUG:tasks.check_counter:Getting stats from l 2024-03-18T15:49:24.222 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af shell --fsid 419c928c-e53c-11ee-95c9-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-18T15:49:24.533 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/mon.c/config 2024-03-18T15:49:24.918 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:24 smithi193 ceph-mon[36702]: pgmap v632: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:25.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:24 smithi137 ceph-mon[35868]: pgmap v632: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:25.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:24 smithi096 ceph-mon[31930]: pgmap v632: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:26.421 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-03-18T15:49:26.421 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-03-18T15:49:26.421 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1969, 2024-03-18T15:49:26.421 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1971, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 8622430, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 626149, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.131014928, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.037671321, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.036418317, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.048416163, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1969, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.045437668, 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000023076 2024-03-18T15:49:26.422 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 7, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 151, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3788, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 332218, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-03-18T15:49:26.423 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.007969532, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.004765947, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.002269883, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000041737, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 150, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.006525079, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000043500 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-03-18T15:49:26.424 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 331520 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 230, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 204, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 903948, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 120638, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 3, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.027375721, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.008275844, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.019928556, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.003767175, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 203, 2024-03-18T15:49:26.425 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.006459591, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000031820 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 903680, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 120320 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-03-18T15:49:26.426 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3417, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.073512551, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000021513 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.427 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-03-18T15:49:26.428 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 1053, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 1053, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 1053, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-03-18T15:49:26.429 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-03-18T15:49:26.430 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-03-18T15:49:26.431 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-03-18T15:49:26.432 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-03-18T15:49:26.433 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 5525, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-03-18T15:49:26.434 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 7, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 12260239, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 12259101, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 5525 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 9, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 1076, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 1067, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 7, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 61, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 54, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 1053, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 2297, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1244, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-03-18T15:49:26.436 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 52068, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 215420 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-03-18T15:49:26.437 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.438 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.439 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.440 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-03-18T15:49:26.441 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.442 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.443 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.444 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.445 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-03-18T15:49:26.446 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-03-18T15:49:26.447 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 1, 2024-03-18T15:49:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1858, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-03-18T15:49:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-03-18T15:49:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-03-18T15:49:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 181320, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 47, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7728, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 163, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-03-18T15:49:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 770744, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 4296, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 1972, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-03-18T15:49:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 1972, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1972, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.916953388, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001479185 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1972, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1973 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 1972, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1972, 2024-03-18T15:49:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1177, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 793, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-03-18T15:49:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-03-18T15:49:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 91, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 5, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 4, 2024-03-18T15:49:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 4, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "osd-slow-ops": { 2024-03-18T15:49:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2202, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 9317273, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2202, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2202, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 9317273, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-03-18T15:49:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1972, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 8065940, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1972, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 8065940, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1972, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1972, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1972, 2024-03-18T15:49:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1972, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55f1ca7886e0": { 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-03-18T15:49:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-03-18T15:49:26.465 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-03-18T15:49:27.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:26 smithi137 ceph-mon[35868]: pgmap v633: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:27.170 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:26 smithi193 ceph-mon[36702]: pgmap v633: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:27.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:26 smithi096 ceph-mon[31930]: pgmap v633: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-03-18T15:49:27.407 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-18T15:49:27.407 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-18T15:49:27.407 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-03-18T15:49:27.408 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-03-18T15:49:27.408 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-03-18T15:49:27.408 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-18T15:49:27.421 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-18T15:49:27.421 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:49:27.421 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:49:27.442 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-03-18T15:49:27.442 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:49:27.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/2177681767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881022b0 msgr2=0x7f0e88108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/2177681767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881022b0 0x7f0e88108b40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f0e80009ca0 tx=0x7f0e8002c0f0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/2177681767 shutdown_connections 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/2177681767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881022b0 0x7f0e88108b40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/2177681767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e881018f0 0x7f0e88101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/2177681767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e88100700 0x7f0e88100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/2177681767 >> 172.21.15.96:0/2177681767 conn(0x7f0e880fbc90 msgr2=0x7f0e880fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/2177681767 shutdown_connections 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/2177681767 wait complete. 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 Processor -- start 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.658+0000 7f0e8e7c6640 1 -- start start 2024-03-18T15:49:27.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e88100700 0x7f0e8806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e881022b0 0x7f0e881a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810c380 con 0x7f0e881018f0 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810c200 con 0x7f0e881022b0 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8810c500 con 0x7f0e88100700 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46304/0 (socket says 172.21.15.96:46304) 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 -- 172.21.15.96:0/4038613013 learned_addr learned my addr 172.21.15.96:0/4038613013 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e87fff640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e88100700 0x7f0e8806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8cd63640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e881022b0 0x7f0e881a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 -- 172.21.15.96:0/4038613013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e88100700 msgr2=0x7f0e8806a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e88100700 0x7f0e8806a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 -- 172.21.15.96:0/4038613013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e881022b0 msgr2=0x7f0e881a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e881022b0 0x7f0e881a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 -- 172.21.15.96:0/4038613013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e881a1960 con 0x7f0e881018f0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e877fe640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e9f0 tx=0x7f0e7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00cd80 con 0x7f0e881018f0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e7c005500 con 0x7f0e881018f0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e881a1bf0 con 0x7f0e881018f0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c010690 con 0x7f0e881018f0 2024-03-18T15:49:27.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.659+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e881a20f0 con 0x7f0e881018f0 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.660+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e881079c0 con 0x7f0e881018f0 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.662+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f0e7c0046c0 con 0x7f0e881018f0 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.662+0000 7f0e857fa640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 0x7f0e700a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.662+0000 7f0e87fff640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 0x7f0e700a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.662+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f0e7c0ceff0 con 0x7f0e881018f0 2024-03-18T15:49:27.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.663+0000 7f0e87fff640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 0x7f0e700a4cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0e88101750 tx=0x7f0e78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:27.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.666+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f0e7c053740 con 0x7f0e881018f0 2024-03-18T15:49:27.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.844+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e8802d380 con 0x7f0e881018f0 2024-03-18T15:49:27.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.846+0000 7f0e857fa640 1 -- 172.21.15.96:0/4038613013 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22715 (secure 0 0 0) 0x7f0e7c09c060 con 0x7f0e881018f0 2024-03-18T15:49:27.847 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:49:27.847 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":91,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:46:48.700289+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 msgr2=0x7f0e700a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 0x7f0e700a4cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0e88101750 tx=0x7f0e78005f90 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 msgr2=0x7f0e8819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e9f0 tx=0x7f0e7c00edf0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 shutdown_connections 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f0e700a2830 0x7f0e700a4cf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e881022b0 0x7f0e881a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e881018f0 0x7f0e8819ec20 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 --2- 172.21.15.96:0/4038613013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e88100700 0x7f0e8806a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 >> 172.21.15.96:0/4038613013 conn(0x7f0e880fbc90 msgr2=0x7f0e88104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:27.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 shutdown_connections 2024-03-18T15:49:27.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.848+0000 7f0e8e7c6640 1 -- 172.21.15.96:0/4038613013 wait complete. 2024-03-18T15:49:27.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:23.693926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:38:02.202561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:38:03.352098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:49:27.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:49:27.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.963+0000 7f35b30dc640 1 -- 172.21.15.96:0/817308535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1018d0 msgr2=0x7f35ac101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.963+0000 7f35b30dc640 1 --2- 172.21.15.96:0/817308535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1018d0 0x7f35ac101d50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3594009900 tx=0x7f359402bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- 172.21.15.96:0/817308535 shutdown_connections 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- 172.21.15.96:0/817308535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac102290 0x7f35ac108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- 172.21.15.96:0/817308535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac1018d0 0x7f35ac101d50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- 172.21.15.96:0/817308535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 0x7f35ac100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- 172.21.15.96:0/817308535 >> 172.21.15.96:0/817308535 conn(0x7f35ac0fbc70 msgr2=0x7f35ac0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- 172.21.15.96:0/817308535 shutdown_connections 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- 172.21.15.96:0/817308535 wait complete. 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 Processor -- start 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- start start 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 0x7f35ac0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 0x7f35ac19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c360 con 0x7f35ac102290 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c1e0 con 0x7f35ac1018d0 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b30dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac10c4e0 con 0x7f35ac1006e0 2024-03-18T15:49:27.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b1679640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b1679640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46312/0 (socket says 172.21.15.96:46312) 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b1679640 1 -- 172.21.15.96:0/2407281267 learned_addr learned my addr 172.21.15.96:0/2407281267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35abfff640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 0x7f35ac19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.964+0000 7f35b0e78640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 0x7f35ac0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 -- 172.21.15.96:0/2407281267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 msgr2=0x7f35ac0694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 0x7f35ac0694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 -- 172.21.15.96:0/2407281267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 msgr2=0x7f35ac19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 0x7f35ac19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 -- 172.21.15.96:0/2407281267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac19f7b0 con 0x7f35ac102290 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35abfff640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 0x7f35ac19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b1679640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9b0 tx=0x7f359c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00cd80 con 0x7f35ac102290 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f359c0035f0 con 0x7f35ac102290 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac19fa40 con 0x7f35ac102290 2024-03-18T15:49:27.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c010670 con 0x7f35ac102290 2024-03-18T15:49:27.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.965+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac19ff10 con 0x7f35ac102290 2024-03-18T15:49:27.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.966+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac107b20 con 0x7f35ac102290 2024-03-18T15:49:27.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.967+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f359c0037a0 con 0x7f35ac102290 2024-03-18T15:49:27.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.968+0000 7f35a9ffb640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 0x7f35980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:27.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.968+0000 7f35b0e78640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 0x7f35980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:27.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.968+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f359c0cedb0 con 0x7f35ac102290 2024-03-18T15:49:27.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.968+0000 7f35b0e78640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 0x7f35980a4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35ac101730 tx=0x7f35a4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:27.972+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f359c09bdc0 con 0x7f35ac102290 2024-03-18T15:49:28.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.148+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35ac0623e0 con 0x7f35ac102290 2024-03-18T15:49:28.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.148+0000 7f35a9ffb640 1 -- 172.21.15.96:0/2407281267 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f359c09b6c0 con 0x7f35ac102290 2024-03-18T15:49:28.149 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 msgr2=0x7f35980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 0x7f35980a4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35ac101730 tx=0x7f35a4009500 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 msgr2=0x7f35ac19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f359c00e9b0 tx=0x7f359c00ee70 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 shutdown_connections 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f35980a2880 0x7f35980a4d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f35ac102290 0x7f35ac19eea0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35ac1018d0 0x7f35ac19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 --2- 172.21.15.96:0/2407281267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac1006e0 0x7f35ac0694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 >> 172.21.15.96:0/2407281267 conn(0x7f35ac0fbc70 msgr2=0x7f35ac104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 shutdown_connections 2024-03-18T15:49:28.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.151+0000 7f35b30dc640 1 -- 172.21.15.96:0/2407281267 wait complete. 2024-03-18T15:49:28.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- 172.21.15.96:0/752211858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4101990 msgr2=0x7f49f41082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/752211858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4101990 0x7f49f41082b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009ca0 tx=0x7f49ec02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- 172.21.15.96:0/752211858 shutdown_connections 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/752211858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4101990 0x7f49f41082b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/752211858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f4100fd0 0x7f49f4101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/752211858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f40ffe80 0x7f49f41002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- 172.21.15.96:0/752211858 >> 172.21.15.96:0/752211858 conn(0x7f49f40fb3d0 msgr2=0x7f49f40fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- 172.21.15.96:0/752211858 shutdown_connections 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- 172.21.15.96:0/752211858 wait complete. 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 Processor -- start 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.266+0000 7f49fc5cd640 1 -- start start 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f40ffe80 0x7f49f410b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4101990 0x7f49f410a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49f410d4e0 con 0x7f49f4100fd0 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49f410d360 con 0x7f49f4101990 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f49f410d660 con 0x7f49f40ffe80 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46340/0 (socket says 172.21.15.96:46340) 2024-03-18T15:49:28.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 -- 172.21.15.96:0/5514069 learned_addr learned my addr 172.21.15.96:0/5514069 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fa369640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f40ffe80 0x7f49f410b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fab6a640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4101990 0x7f49f410a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 -- 172.21.15.96:0/5514069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f40ffe80 msgr2=0x7f49f410b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f40ffe80 0x7f49f410b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 -- 172.21.15.96:0/5514069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4101990 msgr2=0x7f49f410a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4101990 0x7f49f410a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 -- 172.21.15.96:0/5514069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f410aa10 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49f9b68640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f49f000e9c0 tx=0x7f49f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f000cd80 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f41a1ba0 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f00035f0 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0010690 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.267+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a20f0 con 0x7f49f4100fd0 2024-03-18T15:49:28.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.268+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f4107130 con 0x7f49f4100fd0 2024-03-18T15:49:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.269+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f49f00037a0 con 0x7f49f4100fd0 2024-03-18T15:49:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.270+0000 7f49eb7fe640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 0x7f49e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.270+0000 7f49fa369640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 0x7f49e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.270+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f49f00cee70 con 0x7f49f4100fd0 2024-03-18T15:49:28.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.270+0000 7f49fa369640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 0x7f49e00a4cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009bd0 tx=0x7f49e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.274+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f49f009bee0 con 0x7f49f4100fd0 2024-03-18T15:49:28.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.448+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49f40623e0 con 0x7f49f4100fd0 2024-03-18T15:49:28.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.449+0000 7f49eb7fe640 1 -- 172.21.15.96:0/5514069 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f49f009b7e0 con 0x7f49f4100fd0 2024-03-18T15:49:28.450 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 msgr2=0x7f49e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 0x7f49e00a4cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009bd0 tx=0x7f49e4005f90 comp rx=0 tx=0).stop 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 msgr2=0x7f49f4109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f49f000e9c0 tx=0x7f49f000ee80 comp rx=0 tx=0).stop 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 shutdown_connections 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f49e00a2800 0x7f49e00a4cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49f4101990 0x7f49f410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49f4100fd0 0x7f49f4109c40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 --2- 172.21.15.96:0/5514069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f40ffe80 0x7f49f410b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 >> 172.21.15.96:0/5514069 conn(0x7f49f40fb3d0 msgr2=0x7f49f4104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 shutdown_connections 2024-03-18T15:49:28.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.451+0000 7f49fc5cd640 1 -- 172.21.15.96:0/5514069 wait complete. 2024-03-18T15:49:28.462 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:49:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- 172.21.15.96:0/3197717803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30780fffd0 msgr2=0x7f30781003f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 --2- 172.21.15.96:0/3197717803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30780fffd0 0x7f30781003f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f306c009900 tx=0x7f306c02bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- 172.21.15.96:0/3197717803 shutdown_connections 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 --2- 172.21.15.96:0/3197717803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30781019a0 0x7f30781082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 --2- 172.21.15.96:0/3197717803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 0x7f3078101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 --2- 172.21.15.96:0/3197717803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30780fffd0 0x7f30781003f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- 172.21.15.96:0/3197717803 >> 172.21.15.96:0/3197717803 conn(0x7f30780fafa0 msgr2=0x7f30780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- 172.21.15.96:0/3197717803 shutdown_connections 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- 172.21.15.96:0/3197717803 wait complete. 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 Processor -- start 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.566+0000 7f307fdb2640 1 -- start start 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30780fffd0 0x7f3078103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 0x7f3078102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f307810b290 con 0x7f30781019a0 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f307810b110 con 0x7f3078100fe0 2024-03-18T15:49:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f307810b410 con 0x7f30780fffd0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46358/0 (socket says 172.21.15.96:46358) 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 -- 172.21.15.96:0/1101552265 learned_addr learned my addr 172.21.15.96:0/1101552265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307db4e640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30780fffd0 0x7f3078103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307d34d640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 0x7f3078102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 -- 172.21.15.96:0/1101552265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30780fffd0 msgr2=0x7f3078103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30780fffd0 0x7f3078103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 -- 172.21.15.96:0/1101552265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 msgr2=0x7f3078102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 0x7f3078102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 -- 172.21.15.96:0/1101552265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781032f0 con 0x7f30781019a0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307e34f640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f307000ec60 tx=0x7f307000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070004d20 con 0x7f30781019a0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307000ba50 con 0x7f30781019a0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781a1c60 con 0x7f30781019a0 2024-03-18T15:49:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070005740 con 0x7f30781019a0 2024-03-18T15:49:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.567+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a21b0 con 0x7f30781019a0 2024-03-18T15:49:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.568+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30781a23e0 con 0x7f30781019a0 2024-03-18T15:49:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.569+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f30700058e0 con 0x7f30781019a0 2024-03-18T15:49:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.570+0000 7f306affd640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 0x7f30640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.570+0000 7f307db4e640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 0x7f30640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.570+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f30700cf2c0 con 0x7f30781019a0 2024-03-18T15:49:28.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.570+0000 7f307db4e640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 0x7f30640a4cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f306c02efd0 tx=0x7f306c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.573+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f307009c330 con 0x7f30781019a0 2024-03-18T15:49:28.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.749+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f30781a25b0 con 0x7f30781019a0 2024-03-18T15:49:28.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.749+0000 7f306affd640 1 -- 172.21.15.96:0/1101552265 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f307009bc30 con 0x7f30781019a0 2024-03-18T15:49:28.750 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-03-18T15:49:28.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 msgr2=0x7f30640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 0x7f30640a4cc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f306c02efd0 tx=0x7f306c000bc0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 msgr2=0x7f3078102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f307000ec60 tx=0x7f307000c7a0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 shutdown_connections 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f30640a2800 0x7f30640a4cc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30781019a0 0x7f3078102a60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3078100fe0 0x7f3078102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 --2- 172.21.15.96:0/1101552265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30780fffd0 0x7f3078103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 >> 172.21.15.96:0/1101552265 conn(0x7f30780fafa0 msgr2=0x7f3078106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 shutdown_connections 2024-03-18T15:49:28.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.751+0000 7f307fdb2640 1 -- 172.21.15.96:0/1101552265 wait complete. 2024-03-18T15:49:28.761 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:28 smithi096 ceph-mon[31930]: pgmap v634: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:28.762 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4038613013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:28.762 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2407281267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:49:28.762 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:28 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/5514069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:49:28.763 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:49:28.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 -- 172.21.15.96:0/2824517093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67001018f0 msgr2=0x7f6700101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 --2- 172.21.15.96:0/2824517093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67001018f0 0x7f6700101d70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009900 tx=0x7f66f402bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:28.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 -- 172.21.15.96:0/2824517093 shutdown_connections 2024-03-18T15:49:28.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 --2- 172.21.15.96:0/2824517093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001022b0 0x7f6700108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 --2- 172.21.15.96:0/2824517093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67001018f0 0x7f6700101d70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 --2- 172.21.15.96:0/2824517093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700100700 0x7f6700100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 -- 172.21.15.96:0/2824517093 >> 172.21.15.96:0/2824517093 conn(0x7f67000fbc90 msgr2=0x7f67000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 -- 172.21.15.96:0/2824517093 shutdown_connections 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.864+0000 7f6706910640 1 -- 172.21.15.96:0/2824517093 wait complete. 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 Processor -- start 2024-03-18T15:49:28.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 -- start start 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001018f0 0x7f670019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67001022b0 0x7f670019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f670010c360 con 0x7f6700100700 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f670010c1e0 con 0x7f67001022b0 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6706910640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f670010c4e0 con 0x7f67001018f0 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46390/0 (socket says 172.21.15.96:46390) 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 -- 172.21.15.96:0/1267945194 learned_addr learned my addr 172.21.15.96:0/1267945194 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66ff7fe640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001018f0 0x7f670019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f6704ead640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67001022b0 0x7f670019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 -- 172.21.15.96:0/1267945194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001018f0 msgr2=0x7f670019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001018f0 0x7f670019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 -- 172.21.15.96:0/1267945194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67001022b0 msgr2=0x7f670019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67001022b0 0x7f670019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 -- 172.21.15.96:0/1267945194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670019f760 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.865+0000 7f66fffff640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d770 tx=0x7f66f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.866+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.866+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0009c30 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.866+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670019fa50 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.866+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003dd0 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.866+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670019ffc0 con 0x7f6700100700 2024-03-18T15:49:28.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.867+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6700107aa0 con 0x7f6700100700 2024-03-18T15:49:28.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.868+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f66f0003fb0 con 0x7f6700100700 2024-03-18T15:49:28.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.868+0000 7f66fd7fa640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 0x7f66e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:28.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.868+0000 7f66ff7fe640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 0x7f66e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:28.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.868+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f66f00ce680 con 0x7f6700100700 2024-03-18T15:49:28.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.869+0000 7f66ff7fe640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 0x7f66e80a4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f66f402c7b0 tx=0x7f66f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:28.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:28.872+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f66f0009de0 con 0x7f6700100700 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.059+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f670002d370 con 0x7f6700100700 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.059+0000 7f66fd7fa640 1 -- 172.21.15.96:0/1267945194 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+590 (secure 0 0 0) 0x7f66f009b120 con 0x7f6700100700 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2777887346 2024-03-19T15:31:01.360993+0000 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1517993036 2024-03-19T15:31:01.360993+0000 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/2602375712 2024-03-19T15:31:01.360993+0000 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2437645187 2024-03-19T15:31:01.360993+0000 2024-03-18T15:49:29.060 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2398466384 2024-03-19T15:30:34.844871+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3477103440 2024-03-19T15:30:34.844871+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/392915825 2024-03-19T15:30:34.844871+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2538163559 2024-03-19T15:30:34.844871+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/2602375712 2024-03-19T15:31:01.360993+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/392915825 2024-03-19T15:30:34.844871+0000 2024-03-18T15:49:29.061 INFO:teuthology.orchestra.run.smithi096.stderr:listed 10 entries 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 msgr2=0x7f66e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 0x7f66e80a4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f66f402c7b0 tx=0x7f66f4005fd0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 msgr2=0x7f670006a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d770 tx=0x7f66f000dc30 comp rx=0 tx=0).stop 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 shutdown_connections 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f66e80a2880 0x7f66e80a4d40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67001022b0 0x7f670019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67001018f0 0x7f670019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 --2- 172.21.15.96:0/1267945194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6700100700 0x7f670006a980 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 >> 172.21.15.96:0/1267945194 conn(0x7f67000fbc90 msgr2=0x7f6700104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:29.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 shutdown_connections 2024-03-18T15:49:29.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.061+0000 7f6706910640 1 -- 172.21.15.96:0/1267945194 wait complete. 2024-03-18T15:49:29.073 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-18T15:49:29.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:49:29.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-18T15:49:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:28 smithi137 ceph-mon[35868]: pgmap v634: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4038613013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2407281267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:49:29.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:28 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/5514069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:49:29.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:28 smithi193 ceph-mon[36702]: pgmap v634: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 10 op/s 2024-03-18T15:49:29.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4038613013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:29.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2407281267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-18T15:49:29.138 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:28 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/5514069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-18T15:49:29.266 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:49:29.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:49:29.266 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T15:49:29.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:49:29.310 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-03-18T15:49:29.383 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T15:49:29.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:49:29.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:49:29.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:49:29.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-03-18T15:49:29.557 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-18T15:49:29.569 INFO:tasks.cephadm:Teardown begin 2024-03-18T15:49:29.570 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T15:49:29.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- 172.21.15.96:0/116529175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1022b0 msgr2=0x7f53ac108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 --2- 172.21.15.96:0/116529175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1022b0 0x7f53ac108b40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009900 tx=0x7f53a402bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- 172.21.15.96:0/116529175 shutdown_connections 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 --2- 172.21.15.96:0/116529175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1022b0 0x7f53ac108b40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 --2- 172.21.15.96:0/116529175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1018f0 0x7f53ac101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 --2- 172.21.15.96:0/116529175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 0x7f53ac100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- 172.21.15.96:0/116529175 >> 172.21.15.96:0/116529175 conn(0x7f53ac0fbc70 msgr2=0x7f53ac0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- 172.21.15.96:0/116529175 shutdown_connections 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- 172.21.15.96:0/116529175 wait complete. 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 Processor -- start 2024-03-18T15:49:29.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.685+0000 7f53b0819640 1 -- start start 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 0x7f53ac197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1022b0 0x7f53ac19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac10c360 con 0x7f53ac1018f0 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac10c1e0 con 0x7f53ac1022b0 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac10c4e0 con 0x7f53ac100700 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46400/0 (socket says 172.21.15.96:46400) 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 -- 172.21.15.96:0/617110526 learned_addr learned my addr 172.21.15.96:0/617110526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53aa59c640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 0x7f53ac197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53aad9d640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1022b0 0x7f53ac19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 -- 172.21.15.96:0/617110526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 msgr2=0x7f53ac197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 0x7f53ac197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 -- 172.21.15.96:0/617110526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1022b0 msgr2=0x7f53ac19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1022b0 0x7f53ac19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 -- 172.21.15.96:0/617110526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac19f6e0 con 0x7f53ac1018f0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a9d9b640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539400cd80 con 0x7f53ac1018f0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5394010910 con 0x7f53ac1018f0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac19f9d0 con 0x7f53ac1018f0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394018990 con 0x7f53ac1018f0 2024-03-18T15:49:29.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.686+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac19ff20 con 0x7f53ac1018f0 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.687+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ac1a01b0 con 0x7f53ac1018f0 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.688+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f5394010ac0 con 0x7f53ac1018f0 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.689+0000 7f53a37fe640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 0x7f53980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.689+0000 7f53aa59c640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 0x7f53980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.689+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f53940cf1b0 con 0x7f53ac1018f0 2024-03-18T15:49:29.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.690+0000 7f53aa59c640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 0x7f53980a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f53ac101750 tx=0x7f539c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:29.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.693+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f539409bbd0 con 0x7f53ac1018f0 2024-03-18T15:49:29.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.907+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53ac1a0380 con 0x7f53ac1018f0 2024-03-18T15:49:29.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.907+0000 7f53a37fe640 1 -- 172.21.15.96:0/617110526 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11951 (secure 0 0 0) 0x7f539409fc70 con 0x7f53ac1018f0 2024-03-18T15:49:29.908 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:49:29.908 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:49:20.917288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:49:29.909 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 45 2024-03-18T15:49:29.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 msgr2=0x7f53980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 0x7f53980a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f53ac101750 tx=0x7f539c009500 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 msgr2=0x7f53ac198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f539400e9f0 tx=0x7f539400eeb0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 shutdown_connections 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f53980a2800 0x7f53980a4cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53ac1022b0 0x7f53ac19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ac1018f0 0x7f53ac198000 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 --2- 172.21.15.96:0/617110526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53ac100700 0x7f53ac197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 >> 172.21.15.96:0/617110526 conn(0x7f53ac0fbc70 msgr2=0x7f53ac0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 shutdown_connections 2024-03-18T15:49:29.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:29.909+0000 7f53b0819640 1 -- 172.21.15.96:0/617110526 wait complete. 2024-03-18T15:49:29.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1101552265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:49:29.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:29 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1267945194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:49:29.921 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 -- 172.21.15.96:0/4127181509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4074100720 msgr2=0x7f4074100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 --2- 172.21.15.96:0/4127181509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4074100720 0x7f4074100b40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4068009900 tx=0x7f406802bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 -- 172.21.15.96:0/4127181509 shutdown_connections 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 --2- 172.21.15.96:0/4127181509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40741022d0 0x7f4074108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 --2- 172.21.15.96:0/4127181509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074101910 0x7f4074101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 --2- 172.21.15.96:0/4127181509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4074100720 0x7f4074100b40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 -- 172.21.15.96:0/4127181509 >> 172.21.15.96:0/4127181509 conn(0x7f40740fbc90 msgr2=0x7f40740fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 -- 172.21.15.96:0/4127181509 shutdown_connections 2024-03-18T15:49:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.022+0000 7f407af6f640 1 -- 172.21.15.96:0/4127181509 wait complete. 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 Processor -- start 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 -- start start 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074100720 0x7f4074197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4074101910 0x7f4074198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f407410d480 con 0x7f40741022d0 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f407410d300 con 0x7f4074101910 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407af6f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f407410d600 con 0x7f4074100720 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46408/0 (socket says 172.21.15.96:46408) 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 -- 172.21.15.96:0/3432779375 learned_addr learned my addr 172.21.15.96:0/3432779375 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f4073fff640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4074101910 0x7f4074198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f4078d0b640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074100720 0x7f4074197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 -- 172.21.15.96:0/3432779375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074100720 msgr2=0x7f4074197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074100720 0x7f4074197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 -- 172.21.15.96:0/3432779375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4074101910 msgr2=0x7f4074198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4074101910 0x7f4074198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.023+0000 7f407950c640 1 -- 172.21.15.96:0/3432779375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407419f770 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f407950c640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eb10 tx=0x7f406c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c004b20 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406c004cc0 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407419fa60 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c010760 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.024+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407419ff60 con 0x7f40741022d0 2024-03-18T15:49:30.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.025+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4074107a40 con 0x7f40741022d0 2024-03-18T15:49:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.026+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f406c005020 con 0x7f40741022d0 2024-03-18T15:49:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.026+0000 7f4071ffb640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 0x7f40640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.026+0000 7f4078d0b640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 0x7f40640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.026+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f406c0cf010 con 0x7f40741022d0 2024-03-18T15:49:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.027+0000 7f4078d0b640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 0x7f40640a4cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f406802c7b0 tx=0x7f4068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.031+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f406c09c120 con 0x7f40741022d0 2024-03-18T15:49:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1101552265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:49:30.105 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:29 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1267945194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:49:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1101552265' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-18T15:49:30.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:29 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1267945194' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-18T15:49:30.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.245+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f407402d1b0 con 0x7f40741022d0 2024-03-18T15:49:30.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.246+0000 7f4071ffb640 1 -- 172.21.15.96:0/3432779375 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11951 (secure 0 0 0) 0x7f406c09ba20 con 0x7f40741022d0 2024-03-18T15:49:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:49:30.247 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/1518442933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":1518442933},{"type":"v1","addr":"172.21.15.96:6835","nonce":1518442933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2891815137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2891815137},{"type":"v1","addr":"172.21.15.96:6837","nonce":2891815137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-18T15:38:04.167866+0000","modified":"2024-03-18T15:49:20.917288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24493,"mds_1":24485,"mds_2":24472,"mds_3":24460,"mds_4":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6839/36196340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":36196340},{"type":"v1","addr":"172.21.15.96:6839","nonce":36196340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6841/3021384858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3021384858},{"type":"v1","addr":"172.21.15.96:6841","nonce":3021384858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6833/753223976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":753223976},{"type":"v1","addr":"172.21.15.137:6833","nonce":753223976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/3317488579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":3317488579},{"type":"v1","addr":"172.21.15.193:6833","nonce":3317488579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":4,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.137:6837/2776916439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":2776916439},{"type":"v1","addr":"172.21.15.137:6837","nonce":2776916439}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"f","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.193:6835/1114653826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1114653826},{"type":"v1","addr":"172.21.15.193:6835","nonce":1114653826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/2478672579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2478672579},{"type":"v1","addr":"172.21.15.193:6837","nonce":2478672579}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.137:6839/1070718965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":1070718965},{"type":"v1","addr":"172.21.15.137:6839","nonce":1070718965}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.193:6839/1184009925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":1184009925},{"type":"v1","addr":"172.21.15.193:6839","nonce":1184009925}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26960":{"gid":26960,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.137:6835/1669469809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1669469809},{"type":"v1","addr":"172.21.15.137:6835","nonce":1669469809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493,24472,24485,24460,24452]},"id":1}]} 2024-03-18T15:49:30.247 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 45 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 msgr2=0x7f40640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 0x7f40640a4cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f406802c7b0 tx=0x7f4068005fd0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 msgr2=0x7f407419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eb10 tx=0x7f406c00c2d0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 shutdown_connections 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f40640a2800 0x7f40640a4cc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40741022d0 0x7f407419f100 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4074101910 0x7f4074198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 --2- 172.21.15.96:0/3432779375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4074100720 0x7f4074197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 >> 172.21.15.96:0/3432779375 conn(0x7f40740fbc90 msgr2=0x7f4074104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 shutdown_connections 2024-03-18T15:49:30.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.248+0000 7f407af6f640 1 -- 172.21.15.96:0/3432779375 wait complete. 2024-03-18T15:49:30.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:49:30.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2065782201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 msgr2=0x7f58a4108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2065782201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a4108b40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009900 tx=0x7f58a002bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:30.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2065782201 shutdown_connections 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2065782201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a4108b40 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2065782201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 0x7f58a4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2065782201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a4100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.360+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2065782201 >> 172.21.15.96:0/2065782201 conn(0x7f58a40fbc70 msgr2=0x7f58a40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2065782201 shutdown_connections 2024-03-18T15:49:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2065782201 wait complete. 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 Processor -- start 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- start start 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a40694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 0x7f58a419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410c380 con 0x7f58a41022b0 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410c200 con 0x7f58a4100700 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58ab5cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410c500 con 0x7f58a41018f0 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58a9b6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58a9b6a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46420/0 (socket says 172.21.15.96:46420) 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58a9b6a640 1 -- 172.21.15.96:0/2657060177 learned_addr learned my addr 172.21.15.96:0/2657060177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58a9369640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a40694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.361+0000 7f58a8b68640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 0x7f58a419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 -- 172.21.15.96:0/2657060177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 msgr2=0x7f58a419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 0x7f58a419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 -- 172.21.15.96:0/2657060177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 msgr2=0x7f58a40694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a40694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 -- 172.21.15.96:0/2657060177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a419f6f0 con 0x7f58a41022b0 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9369640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a40694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58a9b6a640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0041070 con 0x7f58a41022b0 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a0004450 con 0x7f58a41022b0 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a419f980 con 0x7f58a41022b0 2024-03-18T15:49:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0003e60 con 0x7f58a41022b0 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.362+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a419fec0 con 0x7f58a41022b0 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.363+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a4107a20 con 0x7f58a41022b0 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.364+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f58a0044070 con 0x7f58a41022b0 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.365+0000 7f589a7fc640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 0x7f58900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.365+0000 7f58a9369640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 0x7f58900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.365+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7f58a00f07a0 con 0x7f58a41022b0 2024-03-18T15:49:30.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.365+0000 7f58a9369640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 0x7f58900a4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101750 tx=0x7f589c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.369+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f58a00bd810 con 0x7f58a41022b0 2024-03-18T15:49:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.546+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a40623e0 con 0x7f58a41022b0 2024-03-18T15:49:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.547+0000 7f589a7fc640 1 -- 172.21.15.96:0/2657060177 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22715 (secure 0 0 0) 0x7f58a00bd110 con 0x7f58a41022b0 2024-03-18T15:49:30.548 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-03-18T15:49:30.549 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":91,"fsid":"419c928c-e53c-11ee-95c9-87774f69a715","created":"2024-03-18T15:30:01.553154+0000","modified":"2024-03-18T15:46:48.700289+0000","last_up_change":"2024-03-18T15:37:11.436319+0000","last_in_change":"2024-03-18T15:36:51.273067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:23.693926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:38:02.202561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:38:03.352098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"09dd9fb0-b2af-41c6-bf5b-8c7c01d82677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6803","nonce":3832630015}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6805","nonce":3832630015}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832630015}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3832630015},{"type":"v1","addr":"172.21.15.96:6807","nonce":3832630015}]},"public_addr":"172.21.15.96:6803/3832630015","cluster_addr":"172.21.15.96:6805/3832630015","heartbeat_back_addr":"172.21.15.96:6809/3832630015","heartbeat_front_addr":"172.21.15.96:6807/3832630015","state":["exists","up"]},{"osd":1,"uuid":"72c98ebb-ace5-408f-b919-a69839b4c3b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6801","nonce":3508407501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6803","nonce":3508407501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6807","nonce":3508407501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3508407501},{"type":"v1","addr":"172.21.15.137:6805","nonce":3508407501}]},"public_addr":"172.21.15.137:6801/3508407501","cluster_addr":"172.21.15.137:6803/3508407501","heartbeat_back_addr":"172.21.15.137:6807/3508407501","heartbeat_front_addr":"172.21.15.137:6805/3508407501","state":["exists","up"]},{"osd":2,"uuid":"0139a743-3058-40f5-813f-87cbe0a4d29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6801","nonce":110382863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6803","nonce":110382863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6807","nonce":110382863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":110382863},{"type":"v1","addr":"172.21.15.193:6805","nonce":110382863}]},"public_addr":"172.21.15.193:6801/110382863","cluster_addr":"172.21.15.193:6803/110382863","heartbeat_back_addr":"172.21.15.193:6807/110382863","heartbeat_front_addr":"172.21.15.193:6805/110382863","state":["exists","up"]},{"osd":3,"uuid":"8e02c4cc-033c-4b5f-944c-18fbf2ef29bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6811","nonce":157229437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6813","nonce":157229437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6817","nonce":157229437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":157229437},{"type":"v1","addr":"172.21.15.96:6815","nonce":157229437}]},"public_addr":"172.21.15.96:6811/157229437","cluster_addr":"172.21.15.96:6813/157229437","heartbeat_back_addr":"172.21.15.96:6817/157229437","heartbeat_front_addr":"172.21.15.96:6815/157229437","state":["exists","up"]},{"osd":4,"uuid":"d1281482-ee35-4d0d-be39-618d007e714d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6809","nonce":1772517973}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6811","nonce":1772517973}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6815","nonce":1772517973}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1772517973},{"type":"v1","addr":"172.21.15.137:6813","nonce":1772517973}]},"public_addr":"172.21.15.137:6809/1772517973","cluster_addr":"172.21.15.137:6811/1772517973","heartbeat_back_addr":"172.21.15.137:6815/1772517973","heartbeat_front_addr":"172.21.15.137:6813/1772517973","state":["exists","up"]},{"osd":5,"uuid":"a6e210dc-8cbf-4703-adec-d23138828e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6809","nonce":298847477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6811","nonce":298847477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6815","nonce":298847477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":298847477},{"type":"v1","addr":"172.21.15.193:6813","nonce":298847477}]},"public_addr":"172.21.15.193:6809/298847477","cluster_addr":"172.21.15.193:6811/298847477","heartbeat_back_addr":"172.21.15.193:6815/298847477","heartbeat_front_addr":"172.21.15.193:6813/298847477","state":["exists","up"]},{"osd":6,"uuid":"dd549f23-4564-43ce-8edb-4d5cfad31277","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6819","nonce":3660469146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6821","nonce":3660469146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6825","nonce":3660469146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3660469146},{"type":"v1","addr":"172.21.15.96:6823","nonce":3660469146}]},"public_addr":"172.21.15.96:6819/3660469146","cluster_addr":"172.21.15.96:6821/3660469146","heartbeat_back_addr":"172.21.15.96:6825/3660469146","heartbeat_front_addr":"172.21.15.96:6823/3660469146","state":["exists","up"]},{"osd":7,"uuid":"01fdc4fe-0b84-4109-a831-2b38ba820711","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6817","nonce":1840494271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6819","nonce":1840494271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6823","nonce":1840494271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":1840494271},{"type":"v1","addr":"172.21.15.137:6821","nonce":1840494271}]},"public_addr":"172.21.15.137:6817/1840494271","cluster_addr":"172.21.15.137:6819/1840494271","heartbeat_back_addr":"172.21.15.137:6823/1840494271","heartbeat_front_addr":"172.21.15.137:6821/1840494271","state":["exists","up"]},{"osd":8,"uuid":"db47c340-59a5-4202-8488-b5548f9d18fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6817","nonce":1156737532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6819","nonce":1156737532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6823","nonce":1156737532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1156737532},{"type":"v1","addr":"172.21.15.193:6821","nonce":1156737532}]},"public_addr":"172.21.15.193:6817/1156737532","cluster_addr":"172.21.15.193:6819/1156737532","heartbeat_back_addr":"172.21.15.193:6823/1156737532","heartbeat_front_addr":"172.21.15.193:6821/1156737532","state":["exists","up"]},{"osd":9,"uuid":"214915c3-c5ad-42b8-8c12-04ecc68c6109","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6827","nonce":205742436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6829","nonce":205742436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6833","nonce":205742436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":205742436},{"type":"v1","addr":"172.21.15.96:6831","nonce":205742436}]},"public_addr":"172.21.15.96:6827/205742436","cluster_addr":"172.21.15.96:6829/205742436","heartbeat_back_addr":"172.21.15.96:6833/205742436","heartbeat_front_addr":"172.21.15.96:6831/205742436","state":["exists","up"]},{"osd":10,"uuid":"0d8d5301-49b5-4393-beb1-a3e682d035c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6825","nonce":293078987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6827","nonce":293078987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6831","nonce":293078987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":293078987},{"type":"v1","addr":"172.21.15.137:6829","nonce":293078987}]},"public_addr":"172.21.15.137:6825/293078987","cluster_addr":"172.21.15.137:6827/293078987","heartbeat_back_addr":"172.21.15.137:6831/293078987","heartbeat_front_addr":"172.21.15.137:6829/293078987","state":["exists","up"]},{"osd":11,"uuid":"09d05136-56ba-4ef6-be49-85d156e3c45a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6825","nonce":97980715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6827","nonce":97980715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6831","nonce":97980715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":97980715},{"type":"v1","addr":"172.21.15.193:6829","nonce":97980715}]},"public_addr":"172.21.15.193:6825/97980715","cluster_addr":"172.21.15.193:6827/97980715","heartbeat_back_addr":"172.21.15.193:6831/97980715","heartbeat_front_addr":"172.21.15.193:6829/97980715","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:33.594317+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:32:57.372350+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:21.006423+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:45.478593+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:10.436533+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:34:35.485702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:00.196874+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:26.217404+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:35:51.184353+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:17.233210+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:36:43.332416+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:37:09.272661+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.2","mappings":[{"from":5,"to":7}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.22","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2777887346":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/1517993036":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2437645187":"2024-03-19T15:31:01.360993+0000","172.21.15.96:0/2398466384":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/3477103440":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/392915825":"2024-03-19T15:30:34.844871+0000","172.21.15.96:0/2538163559":"2024-03-19T15:30:34.844871+0000","172.21.15.96:6800/2602375712":"2024-03-19T15:31:01.360993+0000","172.21.15.96:6801/392915825":"2024-03-19T15:30:34.844871+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:49:30.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 msgr2=0x7f58900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 0x7f58900a4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101750 tx=0x7f589c009500 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 msgr2=0x7f58a419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a002c6c0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 shutdown_connections 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f58900a2880 0x7f58900a4d40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a41022b0 0x7f58a419ee30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a41018f0 0x7f58a419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 --2- 172.21.15.96:0/2657060177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58a4100700 0x7f58a40694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.549+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 >> 172.21.15.96:0/2657060177 conn(0x7f58a40fbc70 msgr2=0x7f58a4104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.550+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 shutdown_connections 2024-03-18T15:49:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.550+0000 7f58ab5cd640 1 -- 172.21.15.96:0/2657060177 wait complete. 2024-03-18T15:49:30.562 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- 172.21.15.96:0/1609699507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 msgr2=0x7fb3b0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1609699507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0101d60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fb398009900 tx=0x7fb39802bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- 172.21.15.96:0/1609699507 shutdown_connections 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1609699507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01022a0 0x7fb3b0108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1609699507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0101d60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1609699507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01006f0 0x7fb3b0100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- 172.21.15.96:0/1609699507 >> 172.21.15.96:0/1609699507 conn(0x7fb3b00fbc80 msgr2=0x7fb3b00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- 172.21.15.96:0/1609699507 shutdown_connections 2024-03-18T15:49:30.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- 172.21.15.96:0/1609699507 wait complete. 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 Processor -- start 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.663+0000 7fb3b624b640 1 -- start start 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01006f0 0x7fb3b01958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01022a0 0x7fb3b019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010b270 con 0x7fb3b01018e0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010b0f0 con 0x7fb3b01022a0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b010b3f0 con 0x7fb3b01006f0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3af7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01006f0 0x7fb3b01958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46426/0 (socket says 172.21.15.96:46426) 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 -- 172.21.15.96:0/1720245988 learned_addr learned my addr 172.21.15.96:0/1720245988 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3affff640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01022a0 0x7fb3b019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 -- 172.21.15.96:0/1720245988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01006f0 msgr2=0x7fb3b01958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01006f0 0x7fb3b01958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 -- 172.21.15.96:0/1720245988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01022a0 msgr2=0x7fb3b019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01022a0 0x7fb3b019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 -- 172.21.15.96:0/1720245988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b019d500 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3aeffd640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb398009230 tx=0x7fb39802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398041070 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb398004450 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b019d790 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398003e20 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.664+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b019dcd0 con 0x7fb3b01018e0 2024-03-18T15:49:30.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.665+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b0107b30 con 0x7fb3b01018e0 2024-03-18T15:49:30.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.666+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7fb398044070 con 0x7fb3b01018e0 2024-03-18T15:49:30.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.667+0000 7fb3acff9640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 0x7fb39c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:30.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.667+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8277+0+0 (secure 0 0 0) 0x7fb3980f0910 con 0x7fb3b01018e0 2024-03-18T15:49:30.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.667+0000 7fb3af7fe640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 0x7fb39c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:30.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.667+0000 7fb3af7fe640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 0x7fb39c0a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0101740 tx=0x7fb3a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:30.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.671+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7fb3980bd330 con 0x7fb3b01018e0 2024-03-18T15:49:30.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.889+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb3b002d380 con 0x7fb3b01018e0 2024-03-18T15:49:30.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mon[31930]: pgmap v635: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:30.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/617110526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3432779375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2657060177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.891+0000 7fb3acff9640 1 -- 172.21.15.96:0/1720245988 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fb3980c13d0 con 0x7fb3b01018e0 2024-03-18T15:49:30.892 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.894+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 msgr2=0x7fb39c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.894+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 0x7fb39c0a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0101740 tx=0x7fb3a0005f90 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.894+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 msgr2=0x7fb3b0195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.894+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb398009230 tx=0x7fb39802c650 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 shutdown_connections 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7fb39c0a2880 0x7fb39c0a4d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb3b01022a0 0x7fb3b019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b01018e0 0x7fb3b0195e20 secure :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb398009230 tx=0x7fb39802c650 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 --2- 172.21.15.96:0/1720245988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3b01006f0 0x7fb3b01958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 >> 172.21.15.96:0/1720245988 conn(0x7fb3b00fbc80 msgr2=0x7fb3b0104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 shutdown_connections 2024-03-18T15:49:30.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:30.895+0000 7fb3b624b640 1 -- 172.21.15.96:0/1720245988 wait complete. 2024-03-18T15:49:30.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mon[35868]: pgmap v635: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/617110526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3432779375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2657060177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.956 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mds[65687]: -- 172.21.15.137:0/1213553845 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b54e09fa40 con 0x55b54e073000 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: starting mds.e at 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mds[65998]: -- 172.21.15.137:0/1974716981 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a79fd03a40 con 0x55a79fcd6800 2024-03-18T15:49:30.957 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: starting mds.h at 2024-03-18T15:49:30.958 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.958 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.958 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mds[66323]: -- 172.21.15.137:0/2796959920 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561beca6da40 con 0x561beca40800 2024-03-18T15:49:30.958 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: starting mds.b at 2024-03-18T15:49:30.958 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mon[36702]: pgmap v635: 97 pgs: 97 active+clean; 210 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/617110526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3432779375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2657060177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.959 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mds[67080]: -- 172.21.15.193:0/2379064940 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d4d3753a40 con 0x55d4d3726800 2024-03-18T15:49:30.960 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: starting mds.l at 2024-03-18T15:49:30.960 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:30.960 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:30.960 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mds[68000]: -- 172.21.15.193:0/2080126704 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3449efa40 con 0x55a3449c2c00 2024-03-18T15:49:30.960 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: starting mds.i at 2024-03-18T15:49:31.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 -- 172.21.15.96:0/2411017510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1018e0 msgr2=0x7f18cc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:31.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 --2- 172.21.15.96:0/2411017510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1018e0 0x7f18cc101d60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009900 tx=0x7f18bc02bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:31.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 -- 172.21.15.96:0/2411017510 shutdown_connections 2024-03-18T15:49:31.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 --2- 172.21.15.96:0/2411017510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1022a0 0x7f18cc108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 --2- 172.21.15.96:0/2411017510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1018e0 0x7f18cc101d60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 --2- 172.21.15.96:0/2411017510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 0x7f18cc100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 -- 172.21.15.96:0/2411017510 >> 172.21.15.96:0/2411017510 conn(0x7f18cc0fbc80 msgr2=0x7f18cc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 -- 172.21.15.96:0/2411017510 shutdown_connections 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 -- 172.21.15.96:0/2411017510 wait complete. 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.014+0000 7f18d47f4640 1 Processor -- start 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- start start 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 0x7f18cc199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 0x7f18cc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10c3c0 con 0x7f18cc1022a0 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10c240 con 0x7f18cc1018e0 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc10c540 con 0x7f18cc1006f0 2024-03-18T15:49:31.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46486/0 (socket says 172.21.15.96:46486) 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 -- 172.21.15.96:0/4091737025 learned_addr learned my addr 172.21.15.96:0/4091737025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2590640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 0x7f18cc199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d1d8f640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 0x7f18cc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 -- 172.21.15.96:0/4091737025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 msgr2=0x7f18cc199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 0x7f18cc199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 -- 172.21.15.96:0/4091737025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 msgr2=0x7f18cc19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 0x7f18cc19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 -- 172.21.15.96:0/4091737025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc1a1980 con 0x7f18cc1022a0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d1d8f640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 0x7f18cc19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d2d91640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c970 tx=0x7f18c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8007bb0 con 0x7f18cc1022a0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8007d50 con 0x7f18cc1022a0 2024-03-18T15:49:31.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc1a1c70 con 0x7f18cc1022a0 2024-03-18T15:49:31.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800f450 con 0x7f18cc1022a0 2024-03-18T15:49:31.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.015+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1a2170 con 0x7f18cc1022a0 2024-03-18T15:49:31.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.016+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc1a23a0 con 0x7f18cc1022a0 2024-03-18T15:49:31.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.017+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f18c8004f60 con 0x7f18cc1022a0 2024-03-18T15:49:31.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.018+0000 7f18c37fe640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 0x7f18b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:31.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.018+0000 7f18d2590640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 0x7f18b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:31.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.018+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8707+0+0 (secure 0 0 0) 0x7f18c80ce930 con 0x7f18cc1022a0 2024-03-18T15:49:31.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.019+0000 7f18d2590640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 0x7f18b80a4cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18cc101740 tx=0x7f18c4009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:31.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.023+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f18c8014030 con 0x7f18cc1022a0 2024-03-18T15:49:31.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a[31926]: 2024-03-18T15:49:30.890+0000 7ff66d372700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:49:31.186 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mds[74022]: -- 172.21.15.96:0/2957394655 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633d2c35a40 con 0x5633d2c08800 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: starting mds.a at 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: ignoring --setuser ceph since I am not root 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: ignoring --setgroup ceph since I am not root 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-mds[73666]: -- 172.21.15.96:0/4176847670 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a93b81a40 con 0x558a93b54c00 2024-03-18T15:49:31.187 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:49:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: starting mds.g at 2024-03-18T15:49:31.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.233+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f18cc1a2570 con 0x7f18cc1022a0 2024-03-18T15:49:31.355 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-mds[65367]: -- 172.21.15.137:0/1058406346 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560533e5ba40 con 0x560533e2f000 2024-03-18T15:49:31.356 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:49:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: starting mds.k at 2024-03-18T15:49:31.418 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-mds[67695]: -- 172.21.15.193:0/3573289295 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5c0b9da40 con 0x55d5c0b70c00 2024-03-18T15:49:31.418 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:49:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: starting mds.c at 2024-03-18T15:49:31.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.898+0000 7f18c37fe640 1 -- 172.21.15.96:0/4091737025 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) ==== 104+0+0 (secure 0 0 0) 0x7f18c809b7f0 con 0x7f18cc1022a0 2024-03-18T15:49:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 msgr2=0x7f18b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 0x7f18b80a4cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18cc101740 tx=0x7f18c4009680 comp rx=0 tx=0).stop 2024-03-18T15:49:31.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 msgr2=0x7f18cc1a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18c800c970 tx=0x7f18c800ce30 comp rx=0 tx=0).stop 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 shutdown_connections 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f18b80a2800 0x7f18b80a4cc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18cc1022a0 0x7f18cc1a1310 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18cc1018e0 0x7f18cc19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 --2- 172.21.15.96:0/4091737025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18cc1006f0 0x7f18cc199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 >> 172.21.15.96:0/4091737025 conn(0x7f18cc0fbc80 msgr2=0x7f18cc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 shutdown_connections 2024-03-18T15:49:31.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:31.902+0000 7f18d47f4640 1 -- 172.21.15.96:0/4091737025 wait complete. 2024-03-18T15:49:31.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T15:49:32.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 -- 172.21.15.96:0/113059492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 msgr2=0x7f2384108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 --2- 172.21.15.96:0/113059492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f2384108b20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f237c009ca0 tx=0x7f237c02c0f0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 -- 172.21.15.96:0/113059492 shutdown_connections 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 --2- 172.21.15.96:0/113059492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f2384108b20 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 --2- 172.21.15.96:0/113059492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841018d0 0x7f2384101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 --2- 172.21.15.96:0/113059492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841006e0 0x7f2384100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 -- 172.21.15.96:0/113059492 >> 172.21.15.96:0/113059492 conn(0x7f23840fbc50 msgr2=0x7f23840fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 -- 172.21.15.96:0/113059492 shutdown_connections 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.027+0000 7f238a419640 1 -- 172.21.15.96:0/113059492 wait complete. 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 Processor -- start 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 -- start start 2024-03-18T15:49:32.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841006e0 0x7f23841969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 0x7f2384196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f238410c3d0 con 0x7f2384102290 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f238410c250 con 0x7f23841006e0 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f238a419640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f238410c550 con 0x7f23841018d0 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f23889b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f23889b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46512/0 (socket says 172.21.15.96:46512) 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.028+0000 7f23889b6640 1 -- 172.21.15.96:0/2269885060 learned_addr learned my addr 172.21.15.96:0/2269885060 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f2383fff640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841006e0 0x7f23841969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23837fe640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 0x7f2384196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 -- 172.21.15.96:0/2269885060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 msgr2=0x7f2384196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 0x7f2384196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 -- 172.21.15.96:0/2269885060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841006e0 msgr2=0x7f23841969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:32.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841006e0 0x7f23841969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 -- 172.21.15.96:0/2269885060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238419e610 con 0x7f2384102290 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23837fe640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 0x7f2384196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23889b6640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f237c02c700 tx=0x7f237c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c039040 con 0x7f2384102290 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f237c044070 con 0x7f2384102290 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238419e8a0 con 0x7f2384102290 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0042a0 con 0x7f2384102290 2024-03-18T15:49:32.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.029+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238419ed90 con 0x7f2384102290 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.030+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2384107a00 con 0x7f2384102290 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.031+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f237c032690 con 0x7f2384102290 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.032+0000 7f23817fa640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 0x7f236c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.032+0000 7f2383fff640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 0x7f236c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.032+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8707+0+0 (secure 0 0 0) 0x7f237c0f1200 con 0x7f2384102290 2024-03-18T15:49:32.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.032+0000 7f2383fff640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 0x7f236c0a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2384101730 tx=0x7f2374009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:32.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.037+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f237c0be150 con 0x7f2384102290 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T15:49:32.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:31 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T15:49:32.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:31 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/1720245988' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: osdmap e92: 12 total, 12 up, 12 in 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-18T15:49:32.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:31 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-18T15:49:32.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.211+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f23840623e0 con 0x7f2384102290 2024-03-18T15:49:32.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.900+0000 7f23817fa640 1 -- 172.21.15.96:0/2269885060 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f237c0bda50 con 0x7f2384102290 2024-03-18T15:49:32.901 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-03-18T15:49:32.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 msgr2=0x7f236c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 0x7f236c0a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2384101730 tx=0x7f2374009110 comp rx=0 tx=0).stop 2024-03-18T15:49:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 msgr2=0x7f238419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f237c02c700 tx=0x7f237c032f20 comp rx=0 tx=0).stop 2024-03-18T15:49:32.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 shutdown_connections 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f236c0a2800 0x7f236c0a4cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2384102290 0x7f238419dfa0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23841018d0 0x7f2384196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 --2- 172.21.15.96:0/2269885060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23841006e0 0x7f23841969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 >> 172.21.15.96:0/2269885060 conn(0x7f23840fbc50 msgr2=0x7f2384104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 shutdown_connections 2024-03-18T15:49:32.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:32.903+0000 7f238a419640 1 -- 172.21.15.96:0/2269885060 wait complete. 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: pgmap v637: 97 pgs: 97 active+clean; 210 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: Cluster is now healthy 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6838/2323320193,v1:172.21.15.96:6840/2323320193] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.96:6839/3017388996,v1:172.21.15.96:6841/3017388996] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6836/14997415,v1:172.21.15.137:6837/14997415] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6832/3309349115,v1:172.21.15.137:6833/3309349115] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6832/200092353,v1:172.21.15.193:6835/200092353] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6838/2667844898,v1:172.21.15.137:6839/2667844898] up:boot 2024-03-18T15:49:32.919 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.137:6834/201936042,v1:172.21.15.137:6835/201936042] up:boot 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6838/386803087,v1:172.21.15.193:6839/386803087] up:boot 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6833/2682651173,v1:172.21.15.193:6836/2682651173] up:boot 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: mds.? [v2:172.21.15.193:6834/3968058688,v1:172.21.15.193:6837/3968058688] up:boot 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: fsmap 12 up:standby 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:49:32.920 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:32 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:32.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T15:49:33.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 -- 172.21.15.96:0/3135241822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 msgr2=0x7f3e44105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:33.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3135241822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e44105a60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009900 tx=0x7f3e3402bc30 comp rx=0 tx=0).stop 2024-03-18T15:49:33.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 -- 172.21.15.96:0/3135241822 shutdown_connections 2024-03-18T15:49:33.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3135241822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e44105a60 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3135241822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e44100800 0x7f3e44100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3135241822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e440ffaa0 0x7f3e440ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 -- 172.21.15.96:0/3135241822 >> 172.21.15.96:0/3135241822 conn(0x7f3e440fb3d0 msgr2=0x7f3e440fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 -- 172.21.15.96:0/3135241822 shutdown_connections 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.033+0000 7f3e4af89640 1 -- 172.21.15.96:0/3135241822 wait complete. 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 Processor -- start 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 -- start start 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e440ffaa0 0x7f3e4419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 0x7f3e4419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44113ad0 con 0x7f3e441011c0 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44113950 con 0x7f3e44100800 2024-03-18T15:49:33.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e4af89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44113c50 con 0x7f3e440ffaa0 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46530/0 (socket says 172.21.15.96:46530) 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 -- 172.21.15.96:0/3779559359 learned_addr learned my addr 172.21.15.96:0/3779559359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e48d25640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e440ffaa0 0x7f3e4419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e43fff640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 0x7f3e4419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 -- 172.21.15.96:0/3779559359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e440ffaa0 msgr2=0x7f3e4419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e440ffaa0 0x7f3e4419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 -- 172.21.15.96:0/3779559359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 msgr2=0x7f3e4419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 0x7f3e4419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e49526640 1 -- 172.21.15.96:0/3779559359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a5d30 con 0x7f3e441011c0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.034+0000 7f3e43fff640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 0x7f3e4419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e49526640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3e3402c7b0 tx=0x7f3e34005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34041070 con 0x7f3e441011c0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e340054e0 con 0x7f3e441011c0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034440 con 0x7f3e441011c0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a5fc0 con 0x7f3e441011c0 2024-03-18T15:49:33.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.035+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a6480 con 0x7f3e441011c0 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.036+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e44104850 con 0x7f3e441011c0 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.037+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) ==== 137290+0+0 (secure 0 0 0) 0x7f3e34044070 con 0x7f3e441011c0 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.038+0000 7f3e41ffb640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 0x7f3e2c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.038+0000 7f3e48d25640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 0x7f3e2c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.038+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8182+0+0 (secure 0 0 0) 0x7f3e340f06b0 con 0x7f3e441011c0 2024-03-18T15:49:33.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.038+0000 7f3e48d25640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 0x7f3e2c0a4d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c005fd0 tx=0x7f3e3c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:49:33.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.042+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186149 (secure 0 0 0) 0x7f3e340bd780 con 0x7f3e441011c0 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: pgmap v637: 97 pgs: 97 active+clean; 210 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: Cluster is now healthy 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6838/2323320193,v1:172.21.15.96:6840/2323320193] up:boot 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.96:6839/3017388996,v1:172.21.15.96:6841/3017388996] up:boot 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6836/14997415,v1:172.21.15.137:6837/14997415] up:boot 2024-03-18T15:49:33.106 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6832/3309349115,v1:172.21.15.137:6833/3309349115] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6832/200092353,v1:172.21.15.193:6835/200092353] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6838/2667844898,v1:172.21.15.137:6839/2667844898] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.137:6834/201936042,v1:172.21.15.137:6835/201936042] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6838/386803087,v1:172.21.15.193:6839/386803087] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6833/2682651173,v1:172.21.15.193:6836/2682651173] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: mds.? [v2:172.21.15.193:6834/3968058688,v1:172.21.15.193:6837/3968058688] up:boot 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: fsmap 12 up:standby 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:49:33.107 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:49:33.108 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:32 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: pgmap v637: 97 pgs: 97 active+clean; 210 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 12 op/s 2024-03-18T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-18T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-18T15:49:33.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: Cluster is now healthy 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/4091737025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6838/2323320193,v1:172.21.15.96:6840/2323320193] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.96:6839/3017388996,v1:172.21.15.96:6841/3017388996] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6836/14997415,v1:172.21.15.137:6837/14997415] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6832/3309349115,v1:172.21.15.137:6833/3309349115] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6832/200092353,v1:172.21.15.193:6835/200092353] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6838/2667844898,v1:172.21.15.137:6839/2667844898] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.137:6834/201936042,v1:172.21.15.137:6835/201936042] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6838/386803087,v1:172.21.15.193:6839/386803087] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6833/2682651173,v1:172.21.15.193:6836/2682651173] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: mds.? [v2:172.21.15.193:6834/3968058688,v1:172.21.15.193:6837/3968058688] up:boot 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: fsmap 12 up:standby 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-18T15:49:33.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='mgr.14152 172.21.15.96:0/674870243' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-18T15:49:33.188 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:32 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:33.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.218+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e440623e0 con 0x7f3e441011c0 2024-03-18T15:49:33.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.905+0000 7f3e41ffb640 1 -- 172.21.15.96:0/3779559359 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f3e340bd080 con 0x7f3e441011c0 2024-03-18T15:49:33.906 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-03-18T15:49:33.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.908+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 msgr2=0x7f3e2c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:33.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.908+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 0x7f3e2c0a4d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c005fd0 tx=0x7f3e3c009680 comp rx=0 tx=0).stop 2024-03-18T15:49:33.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.908+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 msgr2=0x7f3e441a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.908+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3e3402c7b0 tx=0x7f3e34005dd0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.908+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 shutdown_connections 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:6800/3036369138,v1:172.21.15.96:6801/3036369138] conn(0x7f3e2c0a2880 0x7f3e2c0a4d40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e441011c0 0x7f3e441a56c0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100800 0x7f3e4419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 --2- 172.21.15.96:0/3779559359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e440ffaa0 0x7f3e4419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 >> 172.21.15.96:0/3779559359 conn(0x7f3e440fb3d0 msgr2=0x7f3e44101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 shutdown_connections 2024-03-18T15:49:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-03-18T15:49:33.909+0000 7f3e4af89640 1 -- 172.21.15.96:0/3779559359 wait complete. 2024-03-18T15:49:33.928 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:49:33.960 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:49:33.994 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:49:34.030 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-18T15:49:34.030 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:49:34.046 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:49:34.061 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-18T15:49:34.085 INFO:tasks.cephadm:Stopping all daemons... 2024-03-18T15:49:34.085 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-18T15:49:34.085 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a 2024-03-18T15:49:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:33 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T15:49:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:33 smithi193 ceph-mon[36702]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T15:49:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:33 smithi193 ceph-mon[36702]: from='client.? 172.21.15.96:0/3779559359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:34.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:33 smithi193 ceph-mon[36702]: pgmap v639: 65 pgs: 65 active+clean; 577 KiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:49:34.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:33 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T15:49:34.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:33 smithi096 ceph-mon[31930]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T15:49:34.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:33 smithi096 ceph-mon[31930]: from='client.? 172.21.15.96:0/3779559359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:34.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:33 smithi096 ceph-mon[31930]: pgmap v639: 65 pgs: 65 active+clean; 577 KiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:49:34.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:34 smithi096 systemd[1]: Stopping Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:34.355 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:33 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/2269885060' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-18T15:49:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:33 smithi137 ceph-mon[35868]: osdmap e93: 12 total, 12 up, 12 in 2024-03-18T15:49:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:33 smithi137 ceph-mon[35868]: from='client.? 172.21.15.96:0/3779559359' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-18T15:49:34.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:33 smithi137 ceph-mon[35868]: pgmap v639: 65 pgs: 65 active+clean; 577 KiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-18T15:49:34.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a[31926]: 2024-03-18T15:49:34.474+0000 7ff672b7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:34.740 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a[31926]: 2024-03-18T15:49:34.474+0000 7ff672b7d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-18T15:49:35.043 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:34 smithi096 podman[100059]: 2024-03-18 15:49:34.73995359 +0000 UTC m=+0.469317876 container died 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-18T15:49:35.369 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 podman[100059]: 2024-03-18 15:49:35.142198516 +0000 UTC m=+0.871562806 container cleanup 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:49:35.369 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 bash[100059]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a 2024-03-18T15:49:35.610 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service' 2024-03-18T15:49:35.634 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 podman[100073]: 2024-03-18 15:49:35.368956942 +0000 UTC m=+0.621118687 container remove 6182999c5ae587155aa284c0adf5de1f8754bb1dfb602275203a9c65e507add2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:49:35.634 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-18T15:49:35.634 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 systemd[1]: Stopped Ceph mon.a for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:49:35.634 INFO:journalctl@ceph.mon.a.smithi096.stdout:Mar 18 15:49:35 smithi096 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.a.service: Consumed 18.720s CPU time. 2024-03-18T15:49:35.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:35.661 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-18T15:49:35.661 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-18T15:49:35.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.b 2024-03-18T15:49:35.990 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:35 smithi137 systemd[1]: Stopping Ceph mon.b for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:36.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b[35864]: 2024-03-18T15:49:35.989+0000 7f5b11057700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:36.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b[35864]: 2024-03-18T15:49:35.989+0000 7f5b11057700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-18T15:49:36.356 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:36 smithi137 podman[70585]: 2024-03-18 15:49:36.142337295 +0000 UTC m=+0.283574867 container died 3b5324c405f851e9acca00622e0aa50efdbb9febb5577fa8b6d6a93f0bf1e479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-18T15:49:36.855 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:36 smithi137 podman[70585]: 2024-03-18 15:49:36.672901454 +0000 UTC m=+0.814139025 container cleanup 3b5324c405f851e9acca00622e0aa50efdbb9febb5577fa8b6d6a93f0bf1e479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-18T15:49:36.856 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:36 smithi137 bash[70585]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b 2024-03-18T15:49:37.146 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.b.service' 2024-03-18T15:49:37.169 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:36 smithi137 podman[70599]: 2024-03-18 15:49:36.916281473 +0000 UTC m=+0.765992730 container remove 3b5324c405f851e9acca00622e0aa50efdbb9febb5577fa8b6d6a93f0bf1e479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-b, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-18T15:49:37.170 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:37 smithi137 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-18T15:49:37.170 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:37 smithi137 systemd[1]: Stopped Ceph mon.b for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:49:37.170 INFO:journalctl@ceph.mon.b.smithi137.stdout:Mar 18 15:49:37 smithi137 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.b.service: Consumed 10.340s CPU time. 2024-03-18T15:49:37.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:37.197 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-18T15:49:37.197 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-18T15:49:37.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.c 2024-03-18T15:49:37.668 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:37 smithi193 systemd[1]: Stopping Ceph mon.c for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:38.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c[36685]: 2024-03-18T15:49:37.684+0000 7f373b9d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:38.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c[36685]: 2024-03-18T15:49:37.684+0000 7f373b9d8700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-18T15:49:38.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:37 smithi193 podman[71887]: 2024-03-18 15:49:37.848327988 +0000 UTC m=+0.405663603 container died 8db2e4fe0093d2bc0f0e3da6680d4188f540a75737aae13d4b25678f8b9791cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-18T15:49:38.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:38 smithi193 podman[71887]: 2024-03-18 15:49:38.243803713 +0000 UTC m=+0.801139351 container cleanup 8db2e4fe0093d2bc0f0e3da6680d4188f540a75737aae13d4b25678f8b9791cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-18T15:49:38.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Mar 18 15:49:38 smithi193 bash[71887]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mon-c 2024-03-18T15:49:38.665 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mon.c.service' 2024-03-18T15:49:38.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:38.711 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-18T15:49:38.711 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-18T15:49:38.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x 2024-03-18T15:49:39.005 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:38 smithi096 systemd[1]: Stopping Ceph mgr.x for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:39.314 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:49:39.003+0000 7faeee5b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:39.315 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x[32163]: 2024-03-18T15:49:39.003+0000 7faeee5b4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-18T15:49:39.315 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 podman[100180]: 2024-03-18 15:49:39.036612824 +0000 UTC m=+0.141497750 container died 258b43739b09b139799659e509d5fd5d4e850184989a3a529d06fff6a68b4b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T15:49:39.595 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 podman[100180]: 2024-03-18 15:49:39.419726425 +0000 UTC m=+0.524611352 container cleanup 258b43739b09b139799659e509d5fd5d4e850184989a3a529d06fff6a68b4b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T15:49:39.596 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 bash[100180]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x 2024-03-18T15:49:39.936 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Mar 18 15:49:39 smithi096 podman[100194]: 2024-03-18 15:49:39.70793214 +0000 UTC m=+0.671591852 container remove 258b43739b09b139799659e509d5fd5d4e850184989a3a529d06fff6a68b4b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d78f5ba07bf18880172c07d4d7b4ac00c9d30af, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-x, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-03-18T15:49:39.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.x.service' 2024-03-18T15:49:40.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:40.020 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-18T15:49:40.021 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-18T15:49:40.021 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.y 2024-03-18T15:49:40.340 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:49:40 smithi137 systemd[1]: Stopping Ceph mgr.y for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:40.606 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:49:40 smithi137 podman[70708]: 2024-03-18 15:49:40.340289083 +0000 UTC m=+0.123938021 container died 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-18T15:49:41.005 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:49:40 smithi137 podman[70708]: 2024-03-18 15:49:40.753331803 +0000 UTC m=+0.536980746 container cleanup 55f7d96e1a076b4f67065fd8259e74055bd4cc921808458f3197dde80eb7467b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:49:41.005 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Mar 18 15:49:40 smithi137 bash[70708]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-y 2024-03-18T15:49:41.220 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.y.service' 2024-03-18T15:49:41.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:41.257 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-18T15:49:41.257 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-18T15:49:41.257 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.z 2024-03-18T15:49:41.603 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:49:41 smithi193 systemd[1]: Stopping Ceph mgr.z for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:41.917 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:49:41 smithi193 podman[72010]: 2024-03-18 15:49:41.603388642 +0000 UTC m=+0.160146358 container died 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-18T15:49:42.168 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:49:42 smithi193 podman[72010]: 2024-03-18 15:49:42.008163578 +0000 UTC m=+0.564921299 container cleanup 9405ffcae5ef3d8ae267d2ad64a02143e0e1756634740c8a62932157135d4f91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-03-18T15:49:42.168 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Mar 18 15:49:42 smithi193 bash[72010]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mgr-z 2024-03-18T15:49:42.425 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mgr.z.service' 2024-03-18T15:49:42.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:42.462 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-18T15:49:42.462 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-18T15:49:42.462 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.0 2024-03-18T15:49:42.760 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:42 smithi096 systemd[1]: Stopping Ceph osd.0 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:43.186 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0[47044]: 2024-03-18T15:49:42.760+0000 7f7f3ac99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:43.186 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0[47044]: 2024-03-18T15:49:42.760+0000 7f7f3ac99700 -1 osd.0 94 *** Got signal Terminated *** 2024-03-18T15:49:43.186 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0[47044]: 2024-03-18T15:49:42.760+0000 7f7f3ac99700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:49:48.403 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:48 smithi096 podman[100304]: 2024-03-18 15:49:48.108911707 +0000 UTC m=+5.464006366 container died 52eb444efd52178c2474d00c80778a318fdea8e2f55e361c60782f06e766a537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:49:48.686 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:48 smithi096 podman[100304]: 2024-03-18 15:49:48.485342655 +0000 UTC m=+5.840437300 container cleanup 52eb444efd52178c2474d00c80778a318fdea8e2f55e361c60782f06e766a537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-18T15:49:48.686 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:48 smithi096 bash[100304]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0 2024-03-18T15:49:49.075 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:48 smithi096 podman[100322]: 2024-03-18 15:49:48.787376185 +0000 UTC m=+0.680382132 container remove 52eb444efd52178c2474d00c80778a318fdea8e2f55e361c60782f06e766a537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-18T15:49:49.357 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:48.978362298 +0000 UTC m=+0.018020623 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:49:49.357 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:49.088363459 +0000 UTC m=+0.128021786 container create 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:49:49.647 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:49.35759416 +0000 UTC m=+0.397252485 container init 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-18T15:49:49.647 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:49.360881257 +0000 UTC m=+0.400539583 container start 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T15:49:49.647 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:49.432672092 +0000 UTC m=+0.472330428 container attach 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T15:49:49.936 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:49 smithi096 podman[100410]: 2024-03-18 15:49:49.647120658 +0000 UTC m=+0.686778984 container died 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:49:50.262 INFO:journalctl@ceph.osd.0.smithi096.stdout:Mar 18 15:49:50 smithi096 podman[100438]: 2024-03-18 15:49:50.236455121 +0000 UTC m=+0.584218419 container remove 230478b0420425e77a0eb0f26ac02c452e0afe86e3f389e43ed1f36a1a9315ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-18T15:49:50.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.0.service' 2024-03-18T15:49:50.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:50.309 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-18T15:49:50.310 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-18T15:49:50.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.1 2024-03-18T15:49:50.606 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:50 smithi137 systemd[1]: Stopping Ceph osd.1 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:51.106 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1[40445]: 2024-03-18T15:49:50.623+0000 7fd3fef3d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:51.106 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1[40445]: 2024-03-18T15:49:50.623+0000 7fd3fef3d700 -1 osd.1 94 *** Got signal Terminated *** 2024-03-18T15:49:51.106 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1[40445]: 2024-03-18T15:49:50.623+0000 7fd3fef3d700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:49:56.409 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:56 smithi137 podman[70832]: 2024-03-18 15:49:56.118882877 +0000 UTC m=+5.618244154 container died b42c20c9ce2f15a856e9a6ca32f868790695d885156b2b75ff3fd517c5db9521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:49:56.842 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:56 smithi137 podman[70832]: 2024-03-18 15:49:56.524219615 +0000 UTC m=+6.023580892 container cleanup b42c20c9ce2f15a856e9a6ca32f868790695d885156b2b75ff3fd517c5db9521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-18T15:49:56.842 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:56 smithi137 bash[70832]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1 2024-03-18T15:49:57.105 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:56 smithi137 podman[70850]: 2024-03-18 15:49:56.842412284 +0000 UTC m=+0.718078044 container remove b42c20c9ce2f15a856e9a6ca32f868790695d885156b2b75ff3fd517c5db9521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T15:49:57.465 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.098359745 +0000 UTC m=+0.017850072 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:49:57.465 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.220606545 +0000 UTC m=+0.140096855 container create d3ade20c6a25668b475ab0e52daaf3686516886e3d7b8a6f29d798d1d53aa6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:49:57.759 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.465324424 +0000 UTC m=+0.384814755 container init d3ade20c6a25668b475ab0e52daaf3686516886e3d7b8a6f29d798d1d53aa6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-18T15:49:57.760 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.468607625 +0000 UTC m=+0.388097947 container start d3ade20c6a25668b475ab0e52daaf3686516886e3d7b8a6f29d798d1d53aa6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:49:57.760 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.573591623 +0000 UTC m=+0.493081951 container attach d3ade20c6a25668b475ab0e52daaf3686516886e3d7b8a6f29d798d1d53aa6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:49:58.105 INFO:journalctl@ceph.osd.1.smithi137.stdout:Mar 18 15:49:57 smithi137 podman[70946]: 2024-03-18 15:49:57.760046061 +0000 UTC m=+0.679536381 container died d3ade20c6a25668b475ab0e52daaf3686516886e3d7b8a6f29d798d1d53aa6ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:49:58.453 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.1.service' 2024-03-18T15:49:58.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:49:58.498 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-18T15:49:58.498 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-18T15:49:58.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.2 2024-03-18T15:49:58.807 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:49:58 smithi193 systemd[1]: Stopping Ceph osd.2 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:49:59.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:49:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2[41865]: 2024-03-18T15:49:58.805+0000 7efcb1eb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:49:59.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:49:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2[41865]: 2024-03-18T15:49:58.805+0000 7efcb1eb7700 -1 osd.2 94 *** Got signal Terminated *** 2024-03-18T15:49:59.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:49:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2[41865]: 2024-03-18T15:49:58.805+0000 7efcb1eb7700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:04.475 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:04 smithi193 podman[72134]: 2024-03-18 15:50:04.195806792 +0000 UTC m=+5.509869505 container died 5d4fe8cd76af3fd711c1ee7b6bb403c32871d4aaecfb0a5f490ed25885244df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-18T15:50:04.867 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:04 smithi193 podman[72134]: 2024-03-18 15:50:04.574182166 +0000 UTC m=+5.888244889 container cleanup 5d4fe8cd76af3fd711c1ee7b6bb403c32871d4aaecfb0a5f490ed25885244df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-18T15:50:04.867 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:04 smithi193 bash[72134]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2 2024-03-18T15:50:05.168 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:04 smithi193 podman[72154]: 2024-03-18 15:50:04.867372795 +0000 UTC m=+0.663172358 container remove 5d4fe8cd76af3fd711c1ee7b6bb403c32871d4aaecfb0a5f490ed25885244df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:50:05.496 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.094703739 +0000 UTC m=+0.017798717 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:05.496 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.23521015 +0000 UTC m=+0.158305155 container create 15c75596c77334631097bb832b101b01f56f5780875bacca9b1264cf45bc4b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True) 2024-03-18T15:50:05.856 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.496215929 +0000 UTC m=+0.419310889 container init 15c75596c77334631097bb832b101b01f56f5780875bacca9b1264cf45bc4b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-18T15:50:05.856 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.499672316 +0000 UTC m=+0.422767274 container start 15c75596c77334631097bb832b101b01f56f5780875bacca9b1264cf45bc4b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-18T15:50:05.856 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.596156547 +0000 UTC m=+0.519251510 container attach 15c75596c77334631097bb832b101b01f56f5780875bacca9b1264cf45bc4b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:50:06.124 INFO:journalctl@ceph.osd.2.smithi193.stdout:Mar 18 15:50:05 smithi193 podman[72245]: 2024-03-18 15:50:05.856757185 +0000 UTC m=+0.779852149 container died 15c75596c77334631097bb832b101b01f56f5780875bacca9b1264cf45bc4b4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-18T15:50:06.455 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.2.service' 2024-03-18T15:50:06.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:06.501 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-18T15:50:06.502 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-18T15:50:06.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.3 2024-03-18T15:50:06.903 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:06 smithi096 systemd[1]: Stopping Ceph osd.3 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:07.186 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:06 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:50:06.945+0000 7f79097b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:07.186 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:06 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:50:06.945+0000 7f79097b0700 -1 osd.3 94 *** Got signal Terminated *** 2024-03-18T15:50:07.186 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:06 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:50:06.945+0000 7f79097b0700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:09.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:09.731+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:09.503+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:10.164 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:09 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:09.763+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:10.436 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:50:10.162+0000 7f7901d9c700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.591211+0000 front 2024-03-18T15:49:44.591321+0000 (oldest deadline 2024-03-18T15:50:09.291365+0000) 2024-03-18T15:50:10.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:10.708+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:10.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:10.498+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:11.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:10 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:10.819+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:11.118 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:10 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:10.777+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:11.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:11.690+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:11.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:11.500+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:12.186 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:11 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3[53856]: 2024-03-18T15:50:11.117+0000 7f7901d9c700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.591211+0000 front 2024-03-18T15:49:44.591321+0000 (oldest deadline 2024-03-18T15:50:09.291365+0000) 2024-03-18T15:50:12.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:11 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:11.743+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:12.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:11 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:11.858+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:12.821 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:12 smithi096 podman[100518]: 2024-03-18 15:50:12.512954891 +0000 UTC m=+5.813392954 container died 72d239c5806619b2f41f3b33ae15175c8605ea9d6ff602299771fda115fd56f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:12.822 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:12 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:12.733+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:12.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:12.646+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:12.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:12.451+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:13.139 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:12 smithi096 podman[100518]: 2024-03-18 15:50:12.920581534 +0000 UTC m=+6.221019595 container cleanup 72d239c5806619b2f41f3b33ae15175c8605ea9d6ff602299771fda115fd56f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:50:13.139 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:12 smithi096 bash[100518]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3 2024-03-18T15:50:13.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:12 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:12.871+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:13.436 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100547]: 2024-03-18 15:50:13.138736117 +0000 UTC m=+0.623563539 container remove 72d239c5806619b2f41f3b33ae15175c8605ea9d6ff602299771fda115fd56f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:50:13.736 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.364226059 +0000 UTC m=+0.017336721 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:13.736 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.473080865 +0000 UTC m=+0.126191520 container create e2d7b4a99a7b65589e1f2958bb33a05a5ed75bf8a122deeba3cab3b123ea041f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307) 2024-03-18T15:50:13.736 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.692524652 +0000 UTC m=+0.345635305 container init e2d7b4a99a7b65589e1f2958bb33a05a5ed75bf8a122deeba3cab3b123ea041f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:50:13.736 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.695680296 +0000 UTC m=+0.348790958 container start e2d7b4a99a7b65589e1f2958bb33a05a5ed75bf8a122deeba3cab3b123ea041f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-18T15:50:13.919 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:13.634+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:13.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:13.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:14.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:13 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:13.781+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:14.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:13 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:13.735+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:14.187 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.775999731 +0000 UTC m=+0.429110402 container attach e2d7b4a99a7b65589e1f2958bb33a05a5ed75bf8a122deeba3cab3b123ea041f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:50:14.187 INFO:journalctl@ceph.osd.3.smithi096.stdout:Mar 18 15:50:13 smithi096 podman[100641]: 2024-03-18 15:50:13.981813359 +0000 UTC m=+0.634924019 container died e2d7b4a99a7b65589e1f2958bb33a05a5ed75bf8a122deeba3cab3b123ea041f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-18T15:50:14.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:13 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:13.875+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:14.607 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.3.service' 2024-03-18T15:50:14.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:14.649 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-18T15:50:14.649 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-18T15:50:14.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.4 2024-03-18T15:50:14.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:14.593+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:14.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:14.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:14.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:14 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:14.746+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:14.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:14 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:14.779+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:14.971 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:14 smithi137 systemd[1]: Stopping Ceph osd.4 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:14.972 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:14.845+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:15.355 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:50:14.971+0000 7fa7e67f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:15.356 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:50:14.971+0000 7fa7e67f1700 -1 osd.4 94 *** Got signal Terminated *** 2024-03-18T15:50:15.356 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:14 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:50:14.971+0000 7fa7e67f1700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:15.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:15 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:15.565+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:15.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:15 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:15.497+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:16.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:15 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:15.744+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:16.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:15 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:15.822+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:16.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:15 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:15.860+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:16.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:16 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:16.518+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:16.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:16 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:16.463+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:17.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:16 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:16.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:17.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:16 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:16.735+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:17.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:16 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:16.797+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:17.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:17.451+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:17.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:17.546+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:17.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:17 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:17.546+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:18.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:17 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:17.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:18.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:17 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:17.707+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:18.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:17 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:17.816+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:18.605 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:18 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:50:18.125+0000 7fa7e2de5700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.525942+0000 front 2024-03-18T15:49:54.525981+0000 (oldest deadline 2024-03-18T15:50:17.926642+0000) 2024-03-18T15:50:18.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:18 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:18.459+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:18.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:18 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:18.558+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:18.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:18 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:18.558+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:18.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:18.777+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:18.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:18.729+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:18.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:18.729+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:19.146 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:18 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:18.887+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:19.146 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:18 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:18.887+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:19.606 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4[47426]: 2024-03-18T15:50:19.145+0000 7fa7e2de5700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.525942+0000 front 2024-03-18T15:49:54.525981+0000 (oldest deadline 2024-03-18T15:50:17.926642+0000) 2024-03-18T15:50:19.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:19 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:19.519+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:19.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:19 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:19.519+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:19.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:19 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:19.420+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:19.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:19 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:19.420+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:20.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:19.840+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:20.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:19 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:19.840+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:20.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:19 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:19.755+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:20.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:19 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:19.727+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:20.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:19 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:19.727+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:20.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:19 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:19.937+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:20.763 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:20 smithi137 podman[71065]: 2024-03-18 15:50:20.500429449 +0000 UTC m=+5.654375130 container died 77252dac31879b0ea62dd9ed8c886604f7334238fee5fb2bbbe965f67a04ddc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-18T15:50:20.900 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:20 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:20.569+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:20.900 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:20 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:20.569+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:20.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:20 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:20.446+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:20.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:20 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:20.446+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:21.105 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:20 smithi137 podman[71065]: 2024-03-18 15:50:20.86239897 +0000 UTC m=+6.016344661 container cleanup 77252dac31879b0ea62dd9ed8c886604f7334238fee5fb2bbbe965f67a04ddc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=) 2024-03-18T15:50:21.106 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:20 smithi137 bash[71065]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4 2024-03-18T15:50:21.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:20 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:20.809+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:21.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:20 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:20.809+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:21.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:20 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:20.899+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:21.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:20 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:20.752+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:21.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:20 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:20.752+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:21.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:20 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:20.799+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:21.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:20 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:20.799+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:21.530 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71082]: 2024-03-18 15:50:21.188774633 +0000 UTC m=+0.686811788 container remove 77252dac31879b0ea62dd9ed8c886604f7334238fee5fb2bbbe965f67a04ddc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:21.531 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:21 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:21.418+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:21.825 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71174]: 2024-03-18 15:50:21.433842803 +0000 UTC m=+0.017992522 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:21.825 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71174]: 2024-03-18 15:50:21.57292739 +0000 UTC m=+0.157077087 container create 1cc8420d61f6953094d042fbfe7bc7129cc9c42f39e3625c95c3289cbcef12ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:21.826 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:21 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:21.769+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:21.826 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:21 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:21.769+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:21.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:21.585+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:21.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:21.585+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:21.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:21.440+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:21.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:21.440+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:22.106 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71174]: 2024-03-18 15:50:21.825730443 +0000 UTC m=+0.409880148 container init 1cc8420d61f6953094d042fbfe7bc7129cc9c42f39e3625c95c3289cbcef12ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-18T15:50:22.106 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71174]: 2024-03-18 15:50:21.828918393 +0000 UTC m=+0.413068106 container start 1cc8420d61f6953094d042fbfe7bc7129cc9c42f39e3625c95c3289cbcef12ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-18T15:50:22.106 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:21 smithi137 podman[71174]: 2024-03-18 15:50:21.933910014 +0000 UTC m=+0.518059733 container attach 1cc8420d61f6953094d042fbfe7bc7129cc9c42f39e3625c95c3289cbcef12ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-18T15:50:22.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:21 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:21.753+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:22.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:21 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:21.753+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:22.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:21 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:21.828+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:22.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:21 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:21.828+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:22.386 INFO:journalctl@ceph.osd.4.smithi137.stdout:Mar 18 15:50:22 smithi137 podman[71174]: 2024-03-18 15:50:22.117901837 +0000 UTC m=+0.702051542 container died 1cc8420d61f6953094d042fbfe7bc7129cc9c42f39e3625c95c3289cbcef12ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T15:50:22.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:21 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:21.928+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:22.653 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:22 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:22.466+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:22.669 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.4.service' 2024-03-18T15:50:22.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:22.710 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-18T15:50:22.710 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-18T15:50:22.710 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.5 2024-03-18T15:50:22.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:22 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:22.538+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:22.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:22 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:22.538+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:22.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:22 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:22.448+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:22.740 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:22 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:22.448+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:23.021 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:22 smithi193 systemd[1]: Stopping Ceph osd.5 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:23.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:22 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:22.940+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:23.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:22 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:22.729+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:23.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:22 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:22.729+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:23.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:22 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:22.763+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:23.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:22 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:22.763+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:23.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:22 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:22.864+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:23.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:22 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:22.864+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:23.418 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:23.021+0000 7f299bf1c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:23.418 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:23.021+0000 7f299bf1c700 -1 osd.5 94 *** Got signal Terminated *** 2024-03-18T15:50:23.418 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:23.021+0000 7f299bf1c700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:23.712 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:23 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:23.429+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:23.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:23.506+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:23.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:23.506+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:23.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:23.448+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:23.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:23.448+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:24.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:23 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:23.711+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:24.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:23 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:23.711+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:24.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:23.756+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:24.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:23.756+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:24.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:23.853+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:24.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:23.853+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:24.415 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:23 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:23.944+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:24.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:24 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:24.415+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:24.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:24 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:24.415+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:24.668 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:24 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:24.533+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:24.668 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:24 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:24.533+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:24.693 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:24 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:24.424+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:25.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:24 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:24.693+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:25.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:24 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:24.693+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:25.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:24 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:24.904+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:25.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:24 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:24.792+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:25.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:24 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:24.792+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:25.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:24 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:24.824+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:25.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:24 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:24.824+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:25.855 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:25 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:25.472+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:25.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:25 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:25.673+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:25.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:25 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:25.673+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:25.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:25 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:25.567+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:25.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:25 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:25.567+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:25.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:25 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:25.423+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:25.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:25 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:25.423+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:26.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:25 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:25.840+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:26.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:25 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:25.840+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:26.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:25 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:25.796+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:26.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:25 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:25.796+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:26.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:25 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:25.796+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:26.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:25 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:25.922+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:26.757 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:26 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:26.472+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:26.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:26 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:26.664+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:26.757 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:26 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:26.664+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:26.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:26.520+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:26.918 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:26.520+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:26.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:26.438+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:26.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:26.438+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:27.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:26 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:26.841+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:27.187 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:26 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:26.841+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:27.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:26 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:26.749+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:27.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:26 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:26.749+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:27.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:26 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:26.749+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:27.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:26.951+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:27.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:26 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:26.951+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:27.770 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:27.461+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:27.771 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:27.461+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:27.771 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:27.702+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:27.771 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:27.702+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:27.771 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:27 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:27.702+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:27.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:27.557+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:47.599771+0000 front 2024-03-18T15:49:47.599718+0000 (oldest deadline 2024-03-18T15:50:09.299802+0000) 2024-03-18T15:50:27.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5[48838]: 2024-03-18T15:50:27.557+0000 7f2994508700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:51.000810+0000 front 2024-03-18T15:49:51.000924+0000 (oldest deadline 2024-03-18T15:50:16.900908+0000) 2024-03-18T15:50:27.912 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:27.478+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:27.912 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:27.478+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:27.913 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:27.478+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:28.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:27.911+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:28.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:27 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:27.911+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:28.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:27.885+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:28.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:27.885+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:28.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:27.885+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:28.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:27.704+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:28.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:27.704+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:28.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:27 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:27.704+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:28.723 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:28 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:28.411+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:28.723 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:28 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:28.411+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:28.747 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:28 smithi193 podman[72363]: 2024-03-18 15:50:28.443922411 +0000 UTC m=+5.533722863 container died bc288e6b0f11e1a7ffb78ad38d8ff4224f7170cfbbd22db2fc5cd9cd0f61b069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5) 2024-03-18T15:50:28.748 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:28 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:28.500+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:28.748 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:28 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:28.500+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:28.748 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:28 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:28.500+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:29.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:28 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:28.722+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:29.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:28 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:28.722+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:29.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:28 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:28.722+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:29.153 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:28 smithi193 podman[72363]: 2024-03-18 15:50:28.85424628 +0000 UTC m=+5.944046729 container cleanup bc288e6b0f11e1a7ffb78ad38d8ff4224f7170cfbbd22db2fc5cd9cd0f61b069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:29.153 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:28 smithi193 bash[72363]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5 2024-03-18T15:50:29.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:28 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:28.937+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:29.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:28 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:28.937+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:29.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:28.839+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:29.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:28.839+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:29.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:28.839+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:29.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:28.707+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:29.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:28.707+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:29.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:28.707+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:29.418 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72380]: 2024-03-18 15:50:29.152755529 +0000 UTC m=+0.709321879 container remove bc288e6b0f11e1a7ffb78ad38d8ff4224f7170cfbbd22db2fc5cd9cd0f61b069 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-18T15:50:29.722 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:29.364+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:29.722 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:29.364+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:29.753 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72468]: 2024-03-18 15:50:29.368845939 +0000 UTC m=+0.017483508 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:29.753 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72468]: 2024-03-18 15:50:29.490597013 +0000 UTC m=+0.139234567 container create cc56f8b3851c22c87c94b9c0efb1166264fc9a7b07f114311f8db830f087efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2) 2024-03-18T15:50:29.753 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:29 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:29.545+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:29.753 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:29 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:29.545+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:29.753 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:29 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:29.545+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:29.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:29.859+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:29.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:29.859+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:29.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:29.859+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:29.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:29.663+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:29.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:29.663+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:29.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:29 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:29.663+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:30.044 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72468]: 2024-03-18 15:50:29.752792596 +0000 UTC m=+0.401430166 container init cc56f8b3851c22c87c94b9c0efb1166264fc9a7b07f114311f8db830f087efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-18T15:50:30.044 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72468]: 2024-03-18 15:50:29.756145115 +0000 UTC m=+0.404782669 container start cc56f8b3851c22c87c94b9c0efb1166264fc9a7b07f114311f8db830f087efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-18T15:50:30.044 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:29 smithi193 podman[72468]: 2024-03-18 15:50:29.843942965 +0000 UTC m=+0.492580543 container attach cc56f8b3851c22c87c94b9c0efb1166264fc9a7b07f114311f8db830f087efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-03-18T15:50:30.044 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:29 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:29.914+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:30.044 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:29 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:29.914+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:30.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:29.721+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:30.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:29.721+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:30.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:29 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:29.721+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:30.358 INFO:journalctl@ceph.osd.5.smithi193.stdout:Mar 18 15:50:30 smithi193 podman[72468]: 2024-03-18 15:50:30.043469673 +0000 UTC m=+0.692107247 container died cc56f8b3851c22c87c94b9c0efb1166264fc9a7b07f114311f8db830f087efd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-18T15:50:30.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:30.529+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:30.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:30.529+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:30.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:30.529+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:30.700 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.5.service' 2024-03-18T15:50:30.758 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:30.360+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:30.758 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:30.360+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:30.782 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:30.782 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-18T15:50:30.782 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-18T15:50:30.783 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.6 2024-03-18T15:50:30.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:30.666+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:30.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:30.666+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:30.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:30.666+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:30.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:30.830+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:30.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:30.830+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:30.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:30 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:30.830+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:30.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:30 smithi096 systemd[1]: Stopping Ceph osd.6 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:31.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:30.757+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:31.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:30.757+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:31.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:30 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:30.757+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:31.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:30.872+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:31.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:30 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:30.872+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:31.436 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.105+0000 7f122cea7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:31.436 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.105+0000 7f122cea7700 -1 osd.6 94 *** Got signal Terminated *** 2024-03-18T15:50:31.436 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.105+0000 7f122cea7700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:31.780 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:31 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:31.382+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:31.780 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:31 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:31.382+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:31.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:31 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:31.512+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:31.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:31 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:31.512+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:31.901 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:31 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:31.512+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:32.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:31 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:31.780+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:32.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:31 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:31.780+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:32.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:31 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:31.780+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:32.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:31 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:31.900+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:32.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:31 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:31.900+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:32.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.808+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:32.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.808+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:32.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:31.808+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:32.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:31.694+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:32.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:31.694+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:32.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:31 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:31.694+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:32.810 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:32 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:32.432+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:32.810 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:32 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:32.432+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:32.913 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:32 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:32.485+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:32.914 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:32 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:32.485+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:32.914 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:32 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:32.485+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:33.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:32 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:32.809+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:33.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:32 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:32.810+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:33.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:32 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:32.810+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:33.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:32 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:32.913+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:33.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:32 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:32.913+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:33.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:32.824+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:33.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:32.824+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:33.186 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:32.824+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:33.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:32.688+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:33.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:32.688+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:33.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:32.688+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:33.802 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:33.439+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:33.802 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:33.439+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:33.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:33.514+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:33.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:33.514+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:33.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:33.514+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:33.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:33.514+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:33.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:33.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:33.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:33.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:33.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:33.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:33.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:33.791+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:33.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:33.791+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:33.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:33.791+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:33.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:33 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:33.791+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.677607+0000 front 2024-03-18T15:50:07.677652+0000 (oldest deadline 2024-03-18T15:50:32.977795+0000) 2024-03-18T15:50:34.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:33.802+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:34.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:33.802+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:34.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:33.802+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:34.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:33 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:33.802+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:34.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:33.944+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:34.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:33 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:33.944+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:34.839 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:34.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:34.839 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:34.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:34.839 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:34.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:34.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:34.526+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:34.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:34.526+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:34.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:34.526+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:34.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:34.526+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:34.646+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:34.646+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:34.646+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:34.798+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:34.798+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:34.936 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:34.798+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:34.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:34 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:34.798+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.677607+0000 front 2024-03-18T15:50:07.677652+0000 (oldest deadline 2024-03-18T15:50:32.977795+0000) 2024-03-18T15:50:35.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:34.838+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:35.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:34.838+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:35.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:34.838+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:35.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:34 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:34.838+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:35.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:34.938+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:35.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:34 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:34.938+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:35.697 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:35.513+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:35.698 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:35.513+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:35.698 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:35.513+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:35.698 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:35.513+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:35.854 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:35.416+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:35.854 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:35.416+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:35.854 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:35.416+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:35.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:35.628+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:35.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:35.628+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:35.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:35.628+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:35.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:35.774+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.673634+0000 front 2024-03-18T15:49:45.673532+0000 (oldest deadline 2024-03-18T15:50:13.274067+0000) 2024-03-18T15:50:35.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:35.774+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.375414+0000 front 2024-03-18T15:49:54.375314+0000 (oldest deadline 2024-03-18T15:50:17.875298+0000) 2024-03-18T15:50:35.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:35.774+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.876771+0000 front 2024-03-18T15:50:01.876750+0000 (oldest deadline 2024-03-18T15:50:27.677273+0000) 2024-03-18T15:50:35.937 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:35 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6[60419]: 2024-03-18T15:50:35.774+0000 7f1225493700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.677607+0000 front 2024-03-18T15:50:07.677652+0000 (oldest deadline 2024-03-18T15:50:32.977795+0000) 2024-03-18T15:50:36.107 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:35.853+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:36.107 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:35.853+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:36.107 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:35.853+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:36.107 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:35 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:35.853+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:36.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:35.940+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:36.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:35.940+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:36.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:35 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:35.940+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:36.731 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:36 smithi096 podman[100747]: 2024-03-18 15:50:36.451385377 +0000 UTC m=+5.476086556 container died 2a6a360f56fb40b2100f3928244e58a1512d8f2f36aeab4a50953cc4d1288c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-18T15:50:36.732 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:36 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:36.622+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:36.732 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:36 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:36.622+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:36.732 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:36 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:36.622+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:36.848 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:36.431+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:36.848 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:36.431+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:36.849 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:36.431+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:36.900 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:36.521+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:36.900 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:36.521+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:36.900 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:36.521+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:36.900 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:36.521+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:37.074 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:36 smithi096 podman[100747]: 2024-03-18 15:50:36.83909593 +0000 UTC m=+5.863797114 container cleanup 2a6a360f56fb40b2100f3928244e58a1512d8f2f36aeab4a50953cc4d1288c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307) 2024-03-18T15:50:37.074 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:36 smithi096 bash[100747]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6 2024-03-18T15:50:37.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:36.847+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:37.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:36.847+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:37.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:36.847+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:37.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:36 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:36.847+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:37.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:36.899+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:37.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:36.899+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:37.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:36 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:36.899+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:37.386 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100765]: 2024-03-18 15:50:37.073679718 +0000 UTC m=+0.621258673 container remove 2a6a360f56fb40b2100f3928244e58a1512d8f2f36aeab4a50953cc4d1288c4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-18T15:50:37.686 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:37 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:37.633+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:37.686 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:37 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:37.633+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:37.686 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:37 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:37.633+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:37.687 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:37 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:37.633+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:37.687 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.288898079 +0000 UTC m=+0.018058573 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:37.687 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.416182849 +0000 UTC m=+0.145343339 container create a6fad3c59765dfaa1f69bc91caf7a2e704e328e7ef1ad1ebd12dca2392e0474d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T15:50:37.687 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.626736992 +0000 UTC m=+0.355897492 container init a6fad3c59765dfaa1f69bc91caf7a2e704e328e7ef1ad1ebd12dca2392e0474d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-18T15:50:37.687 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.629812469 +0000 UTC m=+0.358972967 container start a6fad3c59765dfaa1f69bc91caf7a2e704e328e7ef1ad1ebd12dca2392e0474d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-18T15:50:37.800 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:37.393+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:37.800 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:37.393+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:37.800 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:37.393+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:37.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:37.491+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:37.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:37.491+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:37.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:37.491+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:37.922 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:37.491+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:38.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:37.799+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:38.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:37.799+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:38.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:37.799+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:38.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:37.799+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:38.172 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.735525223 +0000 UTC m=+0.464685729 container attach a6fad3c59765dfaa1f69bc91caf7a2e704e328e7ef1ad1ebd12dca2392e0474d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:50:38.172 INFO:journalctl@ceph.osd.6.smithi096.stdout:Mar 18 15:50:37 smithi096 podman[100854]: 2024-03-18 15:50:37.915718827 +0000 UTC m=+0.644879317 container died a6fad3c59765dfaa1f69bc91caf7a2e704e328e7ef1ad1ebd12dca2392e0474d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-03-18T15:50:38.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:37.941+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:38.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:37.941+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:38.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:37 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:37.941+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:38.460 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.6.service' 2024-03-18T15:50:38.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:38.502 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-18T15:50:38.502 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-18T15:50:38.502 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.7 2024-03-18T15:50:38.686 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:38 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:38.638+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:38.686 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:38 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:38.638+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:38.687 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:38 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:38.638+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:38.687 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:38 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:38.638+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:38.762 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:38.762 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:38.762 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.414+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:38.762 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 systemd[1]: Stopping Ceph osd.7 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:38.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:38.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:38.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:38.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:38.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:38.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:38.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:38.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:39.017 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:38.760+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:39.017 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:38.760+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:39.018 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:38.760+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:39.018 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:38.760+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:39.018 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.838+0000 7f3a49c54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:39.018 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.838+0000 7f3a49c54700 -1 osd.7 94 *** Got signal Terminated *** 2024-03-18T15:50:39.018 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:38 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:38.838+0000 7f3a49c54700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:39.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:38.930+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:39.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:38.930+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:39.418 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:38 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:38.930+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:39.742 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:39.449+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:39.742 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:39.449+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:39.742 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:39.449+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:39.889 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:39.442+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:39.890 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:39.442+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:39.890 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:39.442+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:39.890 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:39.442+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:39.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:39.627+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:39.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:39.627+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:39.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:39.627+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:39.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:39 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:39.627+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:40.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:39.741+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:40.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:39.741+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:40.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:39.741+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:40.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:39 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:39.741+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:40.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:39.889+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:40.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:39.889+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:40.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:39 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:39.889+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:40.485+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:40.485+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:40.485+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:40.714+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:40.714+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:40.714+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:40.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:40.714+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:40.894 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:40.458+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:40.894 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:40.458+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:40.894 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:40.458+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:40.894 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:40.458+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:40.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:40 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:40.631+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:40.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:40 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:40.631+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:40.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:40 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:40.631+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:40.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:40 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:40.631+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:41.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:40.893+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:41.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:40.893+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:41.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:40 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:40.893+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:41.855 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:41.520+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:41.520+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:41.520+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:41.755+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:41.755+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:41.755+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:41.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:41 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:41.755+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:41.877 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:41.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:41.877 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:41.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:41.877 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:41.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:41.877 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:41.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:41.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:41 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:41.637+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:41.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:41 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:41.637+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:41.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:41 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:41.637+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:41.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:41 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:41.637+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:42.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:41.876+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:42.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:41.876+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:42.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:41.876+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:42.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:41 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:41.876+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:42.806 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:42.488+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:42.806 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:42.488+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:42.806 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:42.488+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:42.863 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:42.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:42.863 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:42.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:42.863 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:42.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:42.863 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:42.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:42.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:42.654+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:42.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:42.654+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:42.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:42.654+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:42.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:42.654+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:42.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:42 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:42.654+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:43.105 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:42.805+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:43.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:42.805+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:43.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:42.805+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:43.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:42.805+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:43.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:42 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:42.805+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:43.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:42.862+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:43.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:42.862+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:43.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:42.862+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:43.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:42 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:42.862+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:43.845 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:43.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:43.845 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:43.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:43.845 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:43.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:43.845 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:43.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:43.850 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:43.506+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:49.987676+0000 front 2024-03-18T15:49:49.987702+0000 (oldest deadline 2024-03-18T15:50:21.188354+0000) 2024-03-18T15:50:43.850 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:43.506+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.189042+0000 front 2024-03-18T15:50:01.189026+0000 (oldest deadline 2024-03-18T15:50:27.088922+0000) 2024-03-18T15:50:43.850 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:43.506+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.089408+0000 front 2024-03-18T15:50:07.089424+0000 (oldest deadline 2024-03-18T15:50:34.090155+0000) 2024-03-18T15:50:43.850 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7[54448]: 2024-03-18T15:50:43.506+0000 7f3a42240700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:14.090737+0000 front 2024-03-18T15:50:14.090645+0000 (oldest deadline 2024-03-18T15:50:43.491246+0000) 2024-03-18T15:50:43.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:43 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:43.641+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:43.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:43 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:43.641+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:43.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:43 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:43.641+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:43.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:43 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:43.641+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:43.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:43 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:43.641+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:44.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:43.848+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:44.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:43.848+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:44.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:43.848+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:44.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:43.848+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:44.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:43 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:43.848+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:44.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:43.845+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:44.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:43.845+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:44.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:43.845+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:44.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:43 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:43.845+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:44.566 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:44 smithi137 podman[71291]: 2024-03-18 15:50:44.26186433 +0000 UTC m=+5.561054169 container died 6618498ccffe3d2187125071d83b12d52e03f8094ba8583978b6296123dbd58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-18T15:50:44.822 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:44.475+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:44.822 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:44.475+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:44.822 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:44.475+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:44.822 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:44.475+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:44.822 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:44.475+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:44 smithi137 podman[71291]: 2024-03-18 15:50:44.656214557 +0000 UTC m=+5.955404387 container cleanup 6618498ccffe3d2187125071d83b12d52e03f8094ba8583978b6296123dbd58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:44 smithi137 bash[71291]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:44.811+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:44.811+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:44.811+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:44.811+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:44.859 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:44.811+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:44.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:44 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:44.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:44.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:44 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:44.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:44.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:44 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:44.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:44.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:44 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:44.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:44.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:44 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:44.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:45.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:44.821+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:45.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:44.821+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:45.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:44.821+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:45.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:44 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:44.821+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:45.274 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:44 smithi137 podman[71308]: 2024-03-18 15:50:44.924117309 +0000 UTC m=+0.660202724 container remove 6618498ccffe3d2187125071d83b12d52e03f8094ba8583978b6296123dbd58d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-18T15:50:45.528 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.177139076 +0000 UTC m=+0.017309303 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:45.528 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.308175121 +0000 UTC m=+0.148345333 container create f967cb6205b6beedb4fad1cf67686ad7d39862bc5c32f24d0c99b4cd1a8dfbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-18T15:50:45.528 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.527420035 +0000 UTC m=+0.367590260 container init f967cb6205b6beedb4fad1cf67686ad7d39862bc5c32f24d0c99b4cd1a8dfbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-18T15:50:45.812 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:45.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:45.813 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:45.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:45.813 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:45.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:45.813 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:45.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:45.813 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:45.476+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:45.818 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.530785886 +0000 UTC m=+0.370956106 container start f967cb6205b6beedb4fad1cf67686ad7d39862bc5c32f24d0c99b4cd1a8dfbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-18T15:50:45.818 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.643910065 +0000 UTC m=+0.484080295 container attach f967cb6205b6beedb4fad1cf67686ad7d39862bc5c32f24d0c99b4cd1a8dfbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-18T15:50:45.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:45 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:45.668+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:45.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:45 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:45.668+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:45.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:45 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:45.668+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:45.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:45 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:45.668+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:45.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:45 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:45.668+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:46.105 INFO:journalctl@ceph.osd.7.smithi137.stdout:Mar 18 15:50:45 smithi137 podman[71399]: 2024-03-18 15:50:45.81847218 +0000 UTC m=+0.658642401 container died f967cb6205b6beedb4fad1cf67686ad7d39862bc5c32f24d0c99b4cd1a8dfbb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-18T15:50:46.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:45 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:45.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:46.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:45 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:45.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:46.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:45 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:45.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:46.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:45 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:45.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:46.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:45 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:45.825+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:46.130 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:45.811+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:46.130 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:45.812+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:46.130 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:45.812+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:46.130 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:45 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:45.812+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:46.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.7.service' 2024-03-18T15:50:46.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:46.564 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-18T15:50:46.564 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-18T15:50:46.564 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.8 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 systemd[1]: Stopping Ceph osd.8 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:46.509+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:46.509+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:46.509+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:46.509+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:46.832 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:46.509+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:46.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:46.653+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:46.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:46.653+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:46.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:46.653+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:46.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:46.653+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:46.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:46 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:46.653+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:47.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:46.795+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:47.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:46.795+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:47.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:46.795+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:47.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:46.795+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:47.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:46 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:46.795+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.831+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.831+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.831+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.831+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.909+0000 7f6e082fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.909+0000 7f6e082fe700 -1 osd.8 94 *** Got signal Terminated *** 2024-03-18T15:50:47.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:46 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:46.909+0000 7f6e082fe700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:47.802 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:47.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:47.802 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:47.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:47.802 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:47.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:47.803 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:47.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:47.803 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:47.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:47.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:47 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:47.650+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:47.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:47 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:47.650+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:47.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:47 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:47.650+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:47.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:47 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:47.650+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:47.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:47 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:47.650+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:48.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:47 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:47.818+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:48.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:47 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:47.818+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:48.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:47 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:47.818+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:48.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:47 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:47.818+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:48.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:47 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:47.818+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:48.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:47.801+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:48.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:47.801+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:48.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:47.801+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:48.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:47 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:47.801+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:48.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:48.466+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:48.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:48.466+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:48.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:48.466+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:48.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:48.466+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:48.760 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:48.466+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:48.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:48 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:48.639+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:48.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:48 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:48.639+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:48.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:48 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:48.639+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:48.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:48 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:48.639+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:48.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:48 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:48.639+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:49.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:48.842+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:49.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:48.759+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:49.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:48.759+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:49.169 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:48.759+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:49.169 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:48 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:48.759+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:49.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:49.718+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:49.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:49.718+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:49.718+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:49.718+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:49.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:49 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:49.506+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:49.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:49 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:49.657+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:50.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:49 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:49.857+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:50.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:50.724+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:50.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:50.725+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:50.725+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:50.725+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:50.725+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:25.889427+0000 front 2024-03-18T15:50:25.889451+0000 (oldest deadline 2024-03-18T15:50:50.589552+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:50.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:50 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:50.550+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:50.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:50 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:50.652+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:51.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:50 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:50.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:51.510+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:51.919 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:51.687+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.685328+0000 front 2024-03-18T15:49:53.685236+0000 (oldest deadline 2024-03-18T15:50:18.985324+0000) 2024-03-18T15:50:51.920 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:51.687+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:01.286381+0000 front 2024-03-18T15:50:01.286476+0000 (oldest deadline 2024-03-18T15:50:26.586540+0000) 2024-03-18T15:50:51.920 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:51.687+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.788180+0000 front 2024-03-18T15:50:11.788034+0000 (oldest deadline 2024-03-18T15:50:35.288000+0000) 2024-03-18T15:50:51.920 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:51.687+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:15.288444+0000 front 2024-03-18T15:50:15.288496+0000 (oldest deadline 2024-03-18T15:50:41.188513+0000) 2024-03-18T15:50:51.920 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:51 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8[55848]: 2024-03-18T15:50:51.687+0000 7f6e048f2700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:25.889427+0000 front 2024-03-18T15:50:25.889451+0000 (oldest deadline 2024-03-18T15:50:50.589552+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:51.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:51 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:51.675+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:52.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:51 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:51.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:52 smithi193 podman[72584]: 2024-03-18 15:50:52.321853608 +0000 UTC m=+5.534014191 container died 7b866f576694867e71c16cb9bf9a4bd1bc4c9991e490dcd31b05560a0c49878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:52.651 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:52.477+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:52.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:52 smithi193 podman[72584]: 2024-03-18 15:50:52.650377177 +0000 UTC m=+5.862537770 container cleanup 7b866f576694867e71c16cb9bf9a4bd1bc4c9991e490dcd31b05560a0c49878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:52.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:52 smithi193 bash[72584]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8 2024-03-18T15:50:52.918 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:52 smithi193 podman[72601]: 2024-03-18 15:50:52.885386437 +0000 UTC m=+0.556482050 container remove 7b866f576694867e71c16cb9bf9a4bd1bc4c9991e490dcd31b05560a0c49878b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux ) 2024-03-18T15:50:53.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:53.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:53.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:53.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:53.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:53.187 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:52 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:52.692+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:53.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:52 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:52.917+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:53.613 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.214165345 +0000 UTC m=+0.017888329 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.344426855 +0000 UTC m=+0.148149841 container create b91c9d48ae113351177044900f0d2dd76b895c851e6ccadf40a2c7ff4e0905d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:53.614 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:53 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:53.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:53.905 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.613673819 +0000 UTC m=+0.417396792 container init b91c9d48ae113351177044900f0d2dd76b895c851e6ccadf40a2c7ff4e0905d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-18T15:50:53.906 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.618029369 +0000 UTC m=+0.421752340 container start b91c9d48ae113351177044900f0d2dd76b895c851e6ccadf40a2c7ff4e0905d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.license=GPLv2) 2024-03-18T15:50:53.906 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.696735629 +0000 UTC m=+0.500458615 container attach b91c9d48ae113351177044900f0d2dd76b895c851e6ccadf40a2c7ff4e0905d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:54.168 INFO:journalctl@ceph.osd.8.smithi193.stdout:Mar 18 15:50:53 smithi193 podman[72690]: 2024-03-18 15:50:53.905570276 +0000 UTC m=+0.709293256 container died b91c9d48ae113351177044900f0d2dd76b895c851e6ccadf40a2c7ff4e0905d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:54.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:53 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:53.710+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:54.355 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:54.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:54.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:54.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:54.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:54.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:53 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:53.869+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:54.482 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.8.service' 2024-03-18T15:50:54.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:50:54.524 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-18T15:50:54.525 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-18T15:50:54.525 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.9 2024-03-18T15:50:54.732 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:54.732 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.484+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:54.733 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.484+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:54.733 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.484+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:54.733 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.484+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:54.733 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:54 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:54.484+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 systemd[1]: Stopping Ceph osd.9 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:54.815 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:54.816 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.662+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:55.106 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:54 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:54.846+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:55.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.814+0000 7f16d2b2e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:50:55.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.814+0000 7f16d2b2e700 -1 osd.9 94 *** Got signal Terminated *** 2024-03-18T15:50:55.186 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:54 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:54.814+0000 7f16d2b2e700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:55.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:55 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:55.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:55.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:55 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:55.651+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:56.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:55 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:55.873+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:56.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:56.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:56.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:56.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:56.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:56.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:56.531+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:56.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:56 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:56.643+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:57.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:56 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:56.890+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:57.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:57.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:57.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:57.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:57.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:57.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:57 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:57.483+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:57.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:57.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:57.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:57.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:57.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:57.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:57 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:57.601+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:58.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:57 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:57.915+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:58.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:58.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:58.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:58.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:58.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:58.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:58 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:58.480+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:58.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:58.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:58.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:58.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:58.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:58.937 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:58 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:58.597+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:50:59.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:58 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:58.937+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:50:59.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:50:59.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:50:59.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:50:59.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:50:59.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:50:59.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:50:59 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:50:59.447+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:46.210463+0000 front 2024-03-18T15:49:46.210735+0000 (oldest deadline 2024-03-18T15:50:09.110785+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:55.512571+0000 front 2024-03-18T15:49:55.512328+0000 (oldest deadline 2024-03-18T15:50:20.713169+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:00.713649+0000 front 2024-03-18T15:50:00.713603+0000 (oldest deadline 2024-03-18T15:50:25.413616+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:11.815087+0000 front 2024-03-18T15:50:11.815110+0000 (oldest deadline 2024-03-18T15:50:37.115295+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.115947+0000 front 2024-03-18T15:50:17.116000+0000 (oldest deadline 2024-03-18T15:50:41.815892+0000) 2024-03-18T15:50:59.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:50:59 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9[67168]: 2024-03-18T15:50:59.611+0000 7f16cf122700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.417514+0000 front 2024-03-18T15:50:26.417464+0000 (oldest deadline 2024-03-18T15:50:48.717266+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:00.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:50:59 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:50:59.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:00.490 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:00 smithi096 podman[100973]: 2024-03-18 15:51:00.233568778 +0000 UTC m=+5.520366694 container died 5c266ad593c882a1ebe5b9837d893f883abe1018bdbf69ea16eec4a379fec754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:00.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:00.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:00.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:00.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:00.669 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:00.669 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:00.398+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:00.787 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:00 smithi096 podman[100973]: 2024-03-18 15:51:00.580359384 +0000 UTC m=+5.867157299 container cleanup 5c266ad593c882a1ebe5b9837d893f883abe1018bdbf69ea16eec4a379fec754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:51:00.787 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:00 smithi096 bash[100973]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9 2024-03-18T15:51:01.182 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:00 smithi096 podman[100988]: 2024-03-18 15:51:00.787227179 +0000 UTC m=+0.549014937 container remove 5c266ad593c882a1ebe5b9837d893f883abe1018bdbf69ea16eec4a379fec754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:01.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:00 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:00.922+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:01.436 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.182537858 +0000 UTC m=+0.112585900 container create aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-18T15:51:01.436 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.086997069 +0000 UTC m=+0.017045070 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:51:01.436 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.392931332 +0000 UTC m=+0.322979337 container init aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-18T15:51:01.437 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.396077375 +0000 UTC m=+0.326125374 container start aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:01.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:01 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:01.439+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:01.936 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.468103971 +0000 UTC m=+0.398151981 container attach aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-18T15:51:01.957 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:01 smithi096 podman[101084]: 2024-03-18 15:51:01.676293658 +0000 UTC m=+0.606341649 container died aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-18T15:51:02.226 INFO:journalctl@ceph.osd.9.smithi096.stdout:Mar 18 15:51:02 smithi096 podman[101110]: 2024-03-18 15:51:02.20441627 +0000 UTC m=+0.522872841 container remove aacd7e332a1fd8618a20c20215580ce344b2b4d7dc843f739700a40135698bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:02.231 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.9.service' 2024-03-18T15:51:02.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:02.273 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-18T15:51:02.273 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-18T15:51:02.273 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.10 2024-03-18T15:51:02.303 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:02.303 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:02.303 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:02.303 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:02.303 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:02.304 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:02.304 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:01 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:01.895+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:02.567 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 systemd[1]: Stopping Ceph osd.10 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:02.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:02 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:02.407+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:02.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.566+0000 7fed6cd86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:02.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.566+0000 7fed6cd86700 -1 osd.10 94 *** Got signal Terminated *** 2024-03-18T15:51:02.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.566+0000 7fed6cd86700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:51:03.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:03.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:03.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:03.357 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:03.357 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:03.357 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:03.357 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:02 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:02.886+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:03.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.442+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:03.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.443+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:03.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.443+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:03.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.443+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:03.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.443+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:03.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:03 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:03.443+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:04.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:03 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:03.859+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:04.819 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:04.820 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:04.486+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:05.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:04 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:04.892+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:05.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:05.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:05.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:05.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:05.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:05.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:05.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:05 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:05.482+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:06.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:05 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:05.879+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:06.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:06.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:06 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:06.470+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:45.672963+0000 front 2024-03-18T15:49:45.673040+0000 (oldest deadline 2024-03-18T15:50:10.372942+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:54.473835+0000 front 2024-03-18T15:49:54.473912+0000 (oldest deadline 2024-03-18T15:50:18.574036+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:49:58.574580+0000 front 2024-03-18T15:49:58.574509+0000 (oldest deadline 2024-03-18T15:50:27.375016+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:07.876198+0000 front 2024-03-18T15:50:07.876211+0000 (oldest deadline 2024-03-18T15:50:33.176156+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:19.478213+0000 front 2024-03-18T15:50:19.478196+0000 (oldest deadline 2024-03-18T15:50:42.378406+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:26.479723+0000 front 2024-03-18T15:50:26.479779+0000 (oldest deadline 2024-03-18T15:50:48.779464+0000) 2024-03-18T15:51:07.356 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:06 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10[61070]: 2024-03-18T15:51:06.899+0000 7fed6937a700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-03-18T15:50:35.681477+0000 front 2024-03-18T15:50:35.681596+0000 (oldest deadline 2024-03-18T15:51:01.581435+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:07.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:07.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:07 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:07.481+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:08.236 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:07 smithi137 podman[71517]: 2024-03-18 15:51:07.925462753 +0000 UTC m=+5.459056762 container died 8b8a5554cb5bfc54361a41689e0d597cc95ff96c1f4e967122ab75a79461e216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:51:08.558 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:08 smithi137 podman[71517]: 2024-03-18 15:51:08.340713705 +0000 UTC m=+5.874307714 container cleanup 8b8a5554cb5bfc54361a41689e0d597cc95ff96c1f4e967122ab75a79461e216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-18T15:51:08.558 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:08 smithi137 bash[71517]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10 2024-03-18T15:51:08.856 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:08 smithi137 podman[71534]: 2024-03-18 15:51:08.558292411 +0000 UTC m=+0.623625495 container remove 8b8a5554cb5bfc54361a41689e0d597cc95ff96c1f4e967122ab75a79461e216 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-18T15:51:08.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:08.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:08.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:08.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:08.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:08.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:08.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:08 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:08.505+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:09.137 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:08 smithi137 podman[71625]: 2024-03-18 15:51:08.776861802 +0000 UTC m=+0.017952565 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:51:09.137 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:08 smithi137 podman[71625]: 2024-03-18 15:51:08.900798261 +0000 UTC m=+0.141889042 container create 38b0671e9b93a99a0dcb23ab1b17fe29c8c5ca9e2fb92e269638dc3b796dfcb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-18T15:51:09.137 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:09 smithi137 podman[71625]: 2024-03-18 15:51:09.136722307 +0000 UTC m=+0.377813067 container init 38b0671e9b93a99a0dcb23ab1b17fe29c8c5ca9e2fb92e269638dc3b796dfcb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-03-18T15:51:09.419 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:09 smithi137 podman[71625]: 2024-03-18 15:51:09.139779312 +0000 UTC m=+0.380870075 container start 38b0671e9b93a99a0dcb23ab1b17fe29c8c5ca9e2fb92e269638dc3b796dfcb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:51:09.419 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:09 smithi137 podman[71625]: 2024-03-18 15:51:09.211427495 +0000 UTC m=+0.452518272 container attach 38b0671e9b93a99a0dcb23ab1b17fe29c8c5ca9e2fb92e269638dc3b796dfcb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:51:09.714 INFO:journalctl@ceph.osd.10.smithi137.stdout:Mar 18 15:51:09 smithi137 podman[71625]: 2024-03-18 15:51:09.419044546 +0000 UTC m=+0.660135310 container died 38b0671e9b93a99a0dcb23ab1b17fe29c8c5ca9e2fb92e269638dc3b796dfcb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-18T15:51:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:09.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:09.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:09.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:09 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:09.522+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:10.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.10.service' 2024-03-18T15:51:10.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:10.095 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-18T15:51:10.095 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-18T15:51:10.095 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.11 2024-03-18T15:51:10.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 systemd[1]: Stopping Ceph osd.11 for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:10.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.393+0000 7fb849d3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:10.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.393+0000 7fb849d3e700 -1 osd.11 94 *** Got signal Terminated *** 2024-03-18T15:51:10.395 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.393+0000 7fb849d3e700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:10.668 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:10.669 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:10 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:10.553+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:11.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:11.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:11.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:11.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:11.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:11.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:11.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:11 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:11.554+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:12.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:12.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:12.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:12.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:12.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:12.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:12.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:12.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:12 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:12.602+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-03-18T15:50:49.053417+0000 front 2024-03-18T15:50:49.053437+0000 (oldest deadline 2024-03-18T15:51:12.553358+0000) 2024-03-18T15:51:13.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:13.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:13.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:13.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:13.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:13.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:13.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:13.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:13 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:13.613+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-03-18T15:50:49.053417+0000 front 2024-03-18T15:50:49.053437+0000 (oldest deadline 2024-03-18T15:51:12.553358+0000) 2024-03-18T15:51:14.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-03-18T15:49:44.740396+0000 front 2024-03-18T15:49:44.740527+0000 (oldest deadline 2024-03-18T15:50:08.840421+0000) 2024-03-18T15:51:14.918 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-03-18T15:49:53.442567+0000 front 2024-03-18T15:49:53.442438+0000 (oldest deadline 2024-03-18T15:50:19.342484+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-03-18T15:50:02.744422+0000 front 2024-03-18T15:50:02.744463+0000 (oldest deadline 2024-03-18T15:50:27.444491+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-03-18T15:50:09.145527+0000 front 2024-03-18T15:50:09.145501+0000 (oldest deadline 2024-03-18T15:50:32.645608+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-03-18T15:50:17.346919+0000 front 2024-03-18T15:50:17.346863+0000 (oldest deadline 2024-03-18T15:50:44.347369+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-03-18T15:50:27.248372+0000 front 2024-03-18T15:50:27.248394+0000 (oldest deadline 2024-03-18T15:50:48.849091+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-03-18T15:50:39.152206+0000 front 2024-03-18T15:50:39.152224+0000 (oldest deadline 2024-03-18T15:51:04.452129+0000) 2024-03-18T15:51:14.919 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:14 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11[62782]: 2024-03-18T15:51:14.661+0000 7fb84232a700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-03-18T15:50:49.053417+0000 front 2024-03-18T15:50:49.053437+0000 (oldest deadline 2024-03-18T15:51:12.553358+0000) 2024-03-18T15:51:16.145 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:15 smithi193 podman[72808]: 2024-03-18 15:51:15.850509918 +0000 UTC m=+5.562074627 container died 691eb1b9c7e8295e5f27841e4fee2ac09d4750411ecf0264d93f26f0c708cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-18T15:51:16.145 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72808]: 2024-03-18 15:51:16.143210463 +0000 UTC m=+5.854775170 container cleanup 691eb1b9c7e8295e5f27841e4fee2ac09d4750411ecf0264d93f26f0c708cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-18T15:51:16.418 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 bash[72808]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11 2024-03-18T15:51:16.418 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72825]: 2024-03-18 15:51:16.377627252 +0000 UTC m=+0.525171494 container remove 691eb1b9c7e8295e5f27841e4fee2ac09d4750411ecf0264d93f26f0c708cb88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-18T15:51:16.948 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72913]: 2024-03-18 15:51:16.572271334 +0000 UTC m=+0.017658758 image pull be86dc8cb22b274cd03b7f0beb3c2008b61e12c6812f73ca001a16ae92cb3dae quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513 2024-03-18T15:51:16.948 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72913]: 2024-03-18 15:51:16.686792017 +0000 UTC m=+0.132179459 container create 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:51:16.948 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72913]: 2024-03-18 15:51:16.94763323 +0000 UTC m=+0.393020705 container init 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-18T15:51:17.239 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:16 smithi193 podman[72913]: 2024-03-18 15:51:16.950949562 +0000 UTC m=+0.396336974 container start 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD) 2024-03-18T15:51:17.240 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:17 smithi193 podman[72913]: 2024-03-18 15:51:17.038846552 +0000 UTC m=+0.484233986 container attach 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-18T15:51:17.533 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:17 smithi193 podman[72913]: 2024-03-18 15:51:17.239751814 +0000 UTC m=+0.685139228 container died 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:17.785 INFO:journalctl@ceph.osd.11.smithi193.stdout:Mar 18 15:51:17 smithi193 podman[72939]: 2024-03-18 15:51:17.767027221 +0000 UTC m=+0.522233265 container remove 545c9c95151d2f7259a690ae438e6cb707119c12e975524ad0d05a5d65ff3d79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-osd-11-deactivate, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-18T15:51:17.789 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@osd.11.service' 2024-03-18T15:51:17.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:17.831 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-18T15:51:17.831 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-18T15:51:17.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.a 2024-03-18T15:51:18.148 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:17 smithi096 systemd[1]: Stopping Ceph mds.a for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:18.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: 2024-03-18T15:51:18.146+0000 7ffb7d00d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:18.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:18 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a[74018]: 2024-03-18T15:51:18.146+0000 7ffb7d00d700 -1 mds.a *** got signal Terminated *** 2024-03-18T15:51:22.436 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:22 smithi096 podman[101207]: 2024-03-18 15:51:22.153586974 +0000 UTC m=+4.135928717 container died 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307) 2024-03-18T15:51:22.816 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:22 smithi096 podman[101207]: 2024-03-18 15:51:22.462922572 +0000 UTC m=+4.445264361 container cleanup 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:22.816 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:22 smithi096 bash[101207]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a 2024-03-18T15:51:22.816 INFO:journalctl@ceph.mds.a.smithi096.stdout:Mar 18 15:51:22 smithi096 podman[101221]: 2024-03-18 15:51:22.697238467 +0000 UTC m=+0.533484211 container remove 91c4375e4ee851ef08dc1d54637cf2084c3fa98ac49910ac207230e6870c56a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:51:22.828 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.a.service' 2024-03-18T15:51:22.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:22.865 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-18T15:51:22.865 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-18T15:51:22.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.d 2024-03-18T15:51:23.167 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:22 smithi096 systemd[1]: Stopping Ceph mds.d for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:23.436 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d[73065]: 2024-03-18T15:51:23.165+0000 7fc458469700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:23.436 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:23 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d[73065]: 2024-03-18T15:51:23.166+0000 7fc458469700 -1 mds.d *** got signal Terminated *** 2024-03-18T15:51:27.436 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:27 smithi096 podman[101330]: 2024-03-18 15:51:27.182674893 +0000 UTC m=+4.103079298 container died 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-18T15:51:27.786 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:27 smithi096 podman[101330]: 2024-03-18 15:51:27.574298227 +0000 UTC m=+4.494702641 container cleanup 3ed1546d7ec756d77782b7e5d1e42155841969c6e1932116d7cc92b81e339b2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T15:51:27.786 INFO:journalctl@ceph.mds.d.smithi096.stdout:Mar 18 15:51:27 smithi096 bash[101330]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-d 2024-03-18T15:51:27.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.d.service' 2024-03-18T15:51:27.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:27.946 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-18T15:51:27.946 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-18T15:51:27.946 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.g 2024-03-18T15:51:28.186 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:28 smithi096 systemd[1]: Stopping Ceph mds.g for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:28.686 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: 2024-03-18T15:51:28.279+0000 7f17191a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:28.686 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:28 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g[73662]: 2024-03-18T15:51:28.279+0000 7f17191a6700 -1 mds.g *** got signal Terminated *** 2024-03-18T15:51:31.213 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:30 smithi096 podman[101452]: 2024-03-18 15:51:30.95427114 +0000 UTC m=+2.796798551 container died cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:31.559 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:31 smithi096 podman[101452]: 2024-03-18 15:51:31.316919875 +0000 UTC m=+3.159447291 container cleanup cffe54d2f7fe349b9c5ea75e1da20bd13cd8adb0d20be5895866250c8f6958f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-18T15:51:31.559 INFO:journalctl@ceph.mds.g.smithi096.stdout:Mar 18 15:51:31 smithi096 bash[101452]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-g 2024-03-18T15:51:31.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.g.service' 2024-03-18T15:51:31.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:31.719 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-18T15:51:31.720 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-18T15:51:31.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.j 2024-03-18T15:51:31.936 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:31 smithi096 systemd[1]: Stopping Ceph mds.j for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:32.436 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j[72708]: 2024-03-18T15:51:32.019+0000 7fc6aa34b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:32.436 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:32 smithi096 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j[72708]: 2024-03-18T15:51:32.019+0000 7fc6aa34b700 -1 mds.j *** got signal Terminated *** 2024-03-18T15:51:36.312 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:36 smithi096 podman[101574]: 2024-03-18 15:51:36.032779822 +0000 UTC m=+4.100427756 container died 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:51:36.638 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:36 smithi096 podman[101574]: 2024-03-18 15:51:36.386828504 +0000 UTC m=+4.454476430 container cleanup 3a3f33e5906426b9c01dd50abfd3fac49f10cedb3fda45b017a13ea20490c504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:36.638 INFO:journalctl@ceph.mds.j.smithi096.stdout:Mar 18 15:51:36 smithi096 bash[101574]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-j 2024-03-18T15:51:36.767 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.j.service' 2024-03-18T15:51:36.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:36.801 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-18T15:51:36.801 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-18T15:51:36.801 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.b 2024-03-18T15:51:37.106 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:36 smithi137 systemd[1]: Stopping Ceph mds.b for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:37.373 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: 2024-03-18T15:51:37.137+0000 7fe2b787b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:37.373 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:37 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b[66319]: 2024-03-18T15:51:37.137+0000 7fe2b787b700 -1 mds.b *** got signal Terminated *** 2024-03-18T15:51:39.223 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:38 smithi137 podman[71731]: 2024-03-18 15:51:38.971392198 +0000 UTC m=+1.981025594 container died 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-18T15:51:39.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 podman[71731]: 2024-03-18 15:51:39.305399093 +0000 UTC m=+2.315032488 container cleanup 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T15:51:39.498 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 bash[71731]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b 2024-03-18T15:51:39.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.b.service' 2024-03-18T15:51:39.752 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 podman[71757]: 2024-03-18 15:51:39.497931162 +0000 UTC m=+0.526629542 container remove 669b3735fb36334fc095dfff93d40fc761185c9ad63ed798c172ba2e70912804 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-18T15:51:39.752 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-18T15:51:39.752 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 systemd[1]: Stopped Ceph mds.b for 419c928c-e53c-11ee-95c9-87774f69a715. 2024-03-18T15:51:39.752 INFO:journalctl@ceph.mds.b.smithi137.stdout:Mar 18 15:51:39 smithi137 systemd[1]: ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.b.service: Consumed 3min 58.532s CPU time. 2024-03-18T15:51:39.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:39.766 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-18T15:51:39.766 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-18T15:51:39.767 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.e 2024-03-18T15:51:40.076 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:39 smithi137 systemd[1]: Stopping Ceph mds.e for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:40.356 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: 2024-03-18T15:51:40.074+0000 7f134f291700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:40.356 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:40 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e[65683]: 2024-03-18T15:51:40.074+0000 7f134f291700 -1 mds.e *** got signal Terminated *** 2024-03-18T15:51:43.260 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:42 smithi137 podman[71863]: 2024-03-18 15:51:42.952653841 +0000 UTC m=+2.971229866 container died dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-18T15:51:43.593 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:43 smithi137 podman[71863]: 2024-03-18 15:51:43.353780799 +0000 UTC m=+3.372356756 container cleanup dda81e459d0da3bcaacbf45e0e75c4fc6d431c172909cf4996429001457a6a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-18T15:51:43.593 INFO:journalctl@ceph.mds.e.smithi137.stdout:Mar 18 15:51:43 smithi137 bash[71863]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-e 2024-03-18T15:51:43.713 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.e.service' 2024-03-18T15:51:43.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:43.749 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-18T15:51:43.749 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-18T15:51:43.749 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.h 2024-03-18T15:51:43.855 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:43 smithi137 systemd[1]: Stopping Ceph mds.h for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:44.356 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: 2024-03-18T15:51:44.069+0000 7fe34dc8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:44.356 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:44 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h[65994]: 2024-03-18T15:51:44.069+0000 7fe34dc8b700 -1 mds.h *** got signal Terminated *** 2024-03-18T15:51:47.230 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:46 smithi137 podman[71983]: 2024-03-18 15:51:46.959723282 +0000 UTC m=+2.994109097 container died 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-18T15:51:47.606 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:47 smithi137 podman[71983]: 2024-03-18 15:51:47.311869807 +0000 UTC m=+3.346255665 container cleanup 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:51:47.606 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:47 smithi137 bash[71983]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h 2024-03-18T15:51:47.606 INFO:journalctl@ceph.mds.h.smithi137.stdout:Mar 18 15:51:47 smithi137 podman[71997]: 2024-03-18 15:51:47.479828348 +0000 UTC m=+0.509182912 container remove 860d950f7d70cf4585d5ed81c1d0199af0a6fcbb0e4cf0eb0f70b1f74a6e9e27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-h, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-18T15:51:47.680 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.h.service' 2024-03-18T15:51:47.715 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:47.715 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-18T15:51:47.715 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-18T15:51:47.715 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.k 2024-03-18T15:51:48.032 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:47 smithi137 systemd[1]: Stopping Ceph mds.k for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:48.352 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: 2024-03-18T15:51:48.031+0000 7f72733f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:48.352 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:48 smithi137 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k[65363]: 2024-03-18T15:51:48.031+0000 7f72733f2700 -1 mds.k *** got signal Terminated *** 2024-03-18T15:51:51.356 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:50 smithi137 podman[72105]: 2024-03-18 15:51:50.980270551 +0000 UTC m=+3.048887900 container died 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-18T15:51:51.707 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:51 smithi137 podman[72105]: 2024-03-18 15:51:51.465016182 +0000 UTC m=+3.533633513 container cleanup 8f3ca313288416b54ecff67eaee9fa063a24203fe807195fd280f4aea8f894ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-18T15:51:51.708 INFO:journalctl@ceph.mds.k.smithi137.stdout:Mar 18 15:51:51 smithi137 bash[72105]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-k 2024-03-18T15:51:51.837 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.k.service' 2024-03-18T15:51:51.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:51.871 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-18T15:51:51.871 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-18T15:51:51.871 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.c 2024-03-18T15:51:52.169 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:51 smithi193 systemd[1]: Stopping Ceph mds.c for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:52.435 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: 2024-03-18T15:51:52.332+0000 7f8276154700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:52.435 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:52 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c[67691]: 2024-03-18T15:51:52.332+0000 7f8276154700 -1 mds.c *** got signal Terminated *** 2024-03-18T15:51:55.299 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:54 smithi193 podman[73018]: 2024-03-18 15:51:54.977672851 +0000 UTC m=+2.914277900 container died f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-18T15:51:55.299 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:55 smithi193 podman[73018]: 2024-03-18 15:51:55.298798664 +0000 UTC m=+3.235403718 container cleanup f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T15:51:55.668 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:55 smithi193 bash[73018]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c 2024-03-18T15:51:55.668 INFO:journalctl@ceph.mds.c.smithi193.stdout:Mar 18 15:51:55 smithi193 podman[73044]: 2024-03-18 15:51:55.508926409 +0000 UTC m=+0.525851515 container remove f80cfa8af247314058fa220d4694424da1b6005886758191fd3530ce9e077ca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-03-18T15:51:55.725 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.c.service' 2024-03-18T15:51:55.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:55.764 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-18T15:51:55.764 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-18T15:51:55.764 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.f 2024-03-18T15:51:56.113 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:55 smithi193 systemd[1]: Stopping Ceph mds.f for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:51:56.418 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f[67382]: 2024-03-18T15:51:56.110+0000 7fafa0142700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:51:56.418 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:56 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f[67382]: 2024-03-18T15:51:56.110+0000 7fafa0142700 -1 mds.f *** got signal Terminated *** 2024-03-18T15:51:59.229 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:58 smithi193 podman[73150]: 2024-03-18 15:51:58.973045155 +0000 UTC m=+2.984865292 container died 4fb09f2ce5ec63a3c487b205fd386f48db97b1b2c38b5ccf915f4910a82c1c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-18T15:51:59.668 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:59 smithi193 podman[73150]: 2024-03-18 15:51:59.302677671 +0000 UTC m=+3.314497785 container cleanup 4fb09f2ce5ec63a3c487b205fd386f48db97b1b2c38b5ccf915f4910a82c1c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-18T15:51:59.668 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:59 smithi193 bash[73150]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f 2024-03-18T15:51:59.668 INFO:journalctl@ceph.mds.f.smithi193.stdout:Mar 18 15:51:59 smithi193 podman[73164]: 2024-03-18 15:51:59.478761084 +0000 UTC m=+0.506762877 container remove 4fb09f2ce5ec63a3c487b205fd386f48db97b1b2c38b5ccf915f4910a82c1c0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-18T15:51:59.723 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.f.service' 2024-03-18T15:51:59.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:51:59.759 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-18T15:51:59.759 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-18T15:51:59.760 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.i 2024-03-18T15:52:00.090 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:51:59 smithi193 systemd[1]: Stopping Ceph mds.i for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:52:00.418 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:52:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:52:00.088+0000 7fee8fa34700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:52:00.418 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:52:00 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i[67996]: 2024-03-18T15:52:00.088+0000 7fee8fa34700 -1 mds.i *** got signal Terminated *** 2024-03-18T15:52:03.168 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:52:02 smithi193 podman[73269]: 2024-03-18 15:52:02.967212394 +0000 UTC m=+2.992070051 container died d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-03-18T15:52:03.616 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:52:03 smithi193 podman[73269]: 2024-03-18 15:52:03.381281202 +0000 UTC m=+3.406138842 container cleanup d469ff70a54da0d62d5a03e459ba9b01fb6a41dee8388b9083db7a95ac3651a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-18T15:52:03.616 INFO:journalctl@ceph.mds.i.smithi193.stdout:Mar 18 15:52:03 smithi193 bash[73269]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-i 2024-03-18T15:52:03.836 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.i.service' 2024-03-18T15:52:03.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:52:03.876 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-18T15:52:03.876 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-18T15:52:03.876 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.l 2024-03-18T15:52:04.168 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:03 smithi193 systemd[1]: Stopping Ceph mds.l for 419c928c-e53c-11ee-95c9-87774f69a715... 2024-03-18T15:52:04.668 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: 2024-03-18T15:52:04.193+0000 7f7d64433700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-18T15:52:04.668 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:04 smithi193 ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l[67076]: 2024-03-18T15:52:04.193+0000 7f7d64433700 -1 mds.l *** got signal Terminated *** 2024-03-18T15:52:07.336 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:06 smithi193 podman[73390]: 2024-03-18 15:52:06.971043937 +0000 UTC m=+2.880768923 container died 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240314.183243-3d78f5b) 2024-03-18T15:52:07.668 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:07 smithi193 podman[73390]: 2024-03-18 15:52:07.427093653 +0000 UTC m=+3.336818604 container cleanup 05d3bb9f0d0c089829e305f063a2405f240808d63a883264412ba9eecd927d7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f23dacf9be3936dccb4fcc712e538df27762e9b507b84b5cef1b0402d46f6513, name=ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240314.183243-3d78f5b, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-18T15:52:07.668 INFO:journalctl@ceph.mds.l.smithi193.stdout:Mar 18 15:52:07 smithi193 bash[73390]: ceph-419c928c-e53c-11ee-95c9-87774f69a715-mds-l 2024-03-18T15:52:07.839 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-419c928c-e53c-11ee-95c9-87774f69a715@mds.l.service' 2024-03-18T15:52:07.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-18T15:52:07.873 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-18T15:52:07.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T15:52:08.064 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:52:10.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T15:52:10.635 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:52:13.340 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-18T15:52:13.530 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:52:15.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:52:16.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:52:16.037 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-18T15:52:16.066 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-18T15:52:16.068 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi096/crash 2024-03-18T15:52:16.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash -- . 2024-03-18T15:52:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T15:52:16.099 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-03-18T15:52:16.100 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi137/crash 2024-03-18T15:52:16.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash -- . 2024-03-18T15:52:16.145 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T15:52:16.145 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-03-18T15:52:16.147 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi193/crash 2024-03-18T15:52:16.148 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash -- . 2024-03-18T15:52:16.195 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/419c928c-e53c-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-18T15:52:16.196 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-03-18T15:52:16.197 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-18T15:52:16.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-18T15:52:16.232 INFO:teuthology.orchestra.run.smithi096.stdout:2024-03-18T15:40:03.632795+0000 mon.a (mon.0) 1140 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T15:52:16.233 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-18T15:52:16.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[SEC\]' /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-18T15:52:16.307 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]' /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-18T15:52:16.381 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[WRN\]' /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-18T15:52:16.452 INFO:teuthology.orchestra.run.smithi096.stdout:2024-03-18T15:40:03.632795+0000 mon.a (mon.0) 1140 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-18T15:52:16.453 INFO:tasks.cephadm:Compressing logs... 2024-03-18T15:52:16.453 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:52:16.496 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:52:16.498 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:52:16.519 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T15:52:16.520 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-03-18T15:52:16.520 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.6.log 2024-03-18T15:52:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.a.log 2024-03-18T15:52:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T15:52:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.g.log 2024-03-18T15:52:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.j.log 2024-03-18T15:52:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log: /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-18T15:52:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.x.log 2024-03-18T15:52:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-18T15:52:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.x.log: 90.2% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T15:52:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T15:52:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.0.log: 94.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.j.log.gz 2024-03-18T15:52:16.527 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T15:52:16.527 INFO:teuthology.orchestra.run.smithi137.stderr:: No such file or directory 2024-03-18T15:52:16.527 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.10.log 2024-03-18T15:52:16.528 INFO:teuthology.orchestra.run.smithi096.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T15:52:16.529 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T15:52:16.529 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.b.log 2024-03-18T15:52:16.529 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-18T15:52:16.530 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-18T15:52:16.530 INFO:teuthology.orchestra.run.smithi137.stderr: 92.1%/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.4.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T15:52:16.530 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.k.log 2024-03-18T15:52:16.530 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.h.log 2024-03-18T15:52:16.530 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.7.log 2024-03-18T15:52:16.531 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.e.log 2024-03-18T15:52:16.531 INFO:teuthology.orchestra.run.smithi137.stderr: 90.4%/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.7.log: -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T15:52:16.531 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.y.log 2024-03-18T15:52:16.540 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log 2024-03-18T15:52:16.540 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T15:52:16.540 INFO:teuthology.orchestra.run.smithi096.stderr: 82.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T15:52:16.540 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.9.log 2024-03-18T15:52:16.542 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log: 87.0% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T15:52:16.543 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.d.log 2024-03-18T15:52:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T15:52:16.548 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-18T15:52:16.552 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-18T15:52:16.554 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.l.log 2024-03-18T15:52:16.554 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-18T15:52:16.554 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.i.log 2024-03-18T15:52:16.554 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.11.log 2024-03-18T15:52:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.9.log: 91.0% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T15:52:16.556 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.a.log 2024-03-18T15:52:16.558 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.z.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.8.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.f.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.c.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.c.log 2024-03-18T15:52:16.559 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.z.log.gz 2024-03-18T15:52:16.560 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.d.log.gz 2024-03-18T15:52:16.561 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.y.log: 92.5%gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log 2024-03-18T15:52:16.561 INFO:teuthology.orchestra.run.smithi137.stderr: -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.y.log.gz 2024-03-18T15:52:16.561 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T15:52:16.572 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-18T15:52:16.576 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log: 87.0% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T15:52:16.585 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.b.log 2024-03-18T15:52:16.613 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.b.log: 90.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T15:52:16.639 INFO:teuthology.orchestra.run.smithi193.stderr: 82.6% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-18T15:52:16.661 INFO:teuthology.orchestra.run.smithi193.stderr: 87.0% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-18T15:52:16.731 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.a.log: 94.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.a.log.gz 2024-03-18T15:52:16.778 INFO:teuthology.orchestra.run.smithi096.stderr: 90.1% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mgr.x.log.gz 2024-03-18T15:52:16.849 INFO:teuthology.orchestra.run.smithi193.stderr: 90.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-18T15:52:17.001 INFO:teuthology.orchestra.run.smithi096.stderr: 95.1% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.g.log.gz 2024-03-18T15:52:17.070 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.b.log.gz 2024-03-18T15:52:17.152 INFO:teuthology.orchestra.run.smithi137.stderr: 94.5% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.e.log.gz 2024-03-18T15:52:17.185 INFO:teuthology.orchestra.run.smithi193.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.c.log.gz 2024-03-18T15:52:17.366 INFO:teuthology.orchestra.run.smithi096.stderr: 90.5% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mon.a.log.gz 2024-03-18T15:52:17.427 INFO:teuthology.orchestra.run.smithi193.stderr: 95.1% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.l.log.gz 2024-03-18T15:52:18.081 INFO:teuthology.orchestra.run.smithi193.stderr: 90.4% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-18T15:52:22.836 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.9.log.gz 2024-03-18T15:52:25.301 INFO:teuthology.orchestra.run.smithi137.stderr: 94.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.b.log.gz 2024-03-18T15:52:26.230 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.6.log.gz 2024-03-18T15:52:26.669 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-18T15:52:27.029 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-18T15:52:27.031 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-03-18T15:52:27.031 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m10.523s 2024-03-18T15:52:27.031 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m37.194s 2024-03-18T15:52:27.031 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m1.193s 2024-03-18T15:52:27.140 INFO:teuthology.orchestra.run.smithi137.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-18T15:52:27.342 INFO:teuthology.orchestra.run.smithi193.stderr: 94.2% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.f.log.gz 2024-03-18T15:52:27.572 INFO:teuthology.orchestra.run.smithi137.stderr: 94.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.h.log.gz 2024-03-18T15:52:27.858 INFO:teuthology.orchestra.run.smithi137.stderr: 91.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-18T15:52:29.209 INFO:teuthology.orchestra.run.smithi137.stderr: 91.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.10.log.gz 2024-03-18T15:52:30.399 INFO:teuthology.orchestra.run.smithi137.stderr: 91.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.7.log.gz 2024-03-18T15:52:31.383 INFO:teuthology.orchestra.run.smithi193.stderr: 94.4% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.i.log.gz 2024-03-18T15:52:32.479 INFO:teuthology.orchestra.run.smithi137.stderr: 94.9% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.k.log.gz 2024-03-18T15:52:32.481 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-18T15:52:32.482 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m15.969s 2024-03-18T15:52:32.482 INFO:teuthology.orchestra.run.smithi137.stderr:user 1m22.332s 2024-03-18T15:52:32.482 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m2.360s 2024-03-18T15:52:33.597 INFO:teuthology.orchestra.run.smithi193.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-18T15:52:34.615 INFO:teuthology.orchestra.run.smithi193.stderr: 91.7% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.8.log.gz 2024-03-18T15:52:35.000 INFO:teuthology.orchestra.run.smithi193.stderr: 91.8% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-18T15:52:36.133 INFO:teuthology.orchestra.run.smithi193.stderr: 92.1% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-osd.11.log.gz 2024-03-18T15:55:55.407 INFO:teuthology.orchestra.run.smithi193.stderr: 93.0% -- replaced with /var/log/ceph/419c928c-e53c-11ee-95c9-87774f69a715/ceph-mds.c.log.gz 2024-03-18T15:55:55.409 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-03-18T15:55:55.409 INFO:teuthology.orchestra.run.smithi193.stderr:real 3m38.895s 2024-03-18T15:55:55.409 INFO:teuthology.orchestra.run.smithi193.stderr:user 4m16.251s 2024-03-18T15:55:55.409 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m8.075s 2024-03-18T15:55:55.409 INFO:tasks.cephadm:Archiving logs... 2024-03-18T15:55:55.410 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi096/log 2024-03-18T15:55:55.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:56:01.536 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi137/log 2024-03-18T15:56:01.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:56:11.080 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi193/log 2024-03-18T15:56:11.081 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:56:50.139 INFO:tasks.cephadm:Removing cluster... 2024-03-18T15:56:50.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force 2024-03-18T15:56:50.333 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:56:50.911 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force 2024-03-18T15:56:51.103 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:56:51.445 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 419c928c-e53c-11ee-95c9-87774f69a715 --force 2024-03-18T15:56:51.635 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 419c928c-e53c-11ee-95c9-87774f69a715 2024-03-18T15:56:52.140 INFO:tasks.cephadm:Removing cephadm ... 2024-03-18T15:56:52.140 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T15:56:52.157 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T15:56:52.171 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-18T15:56:52.186 INFO:tasks.cephadm:Teardown complete 2024-03-18T15:56:52.187 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T15:56:52.197 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T15:56:52.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:56:52.200 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:56:52.214 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:56:52.328 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T15:56:52.338 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T15:56:52.339 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:56:52.341 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:56:52.343 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:56:52.353 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:56:52.355 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:56:52.356 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +2355us[+2355us] +/- 117ms 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -1499us[-1471us] +/- 66ms 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +316us[ +316us] +/- 87ms 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:56:52.357 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-03-18T15:56:52.359 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:56:52.359 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-18T15:56:52.359 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +2363us[+2376us] +/- 117ms 2024-03-18T15:56:52.359 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -1489us[-1477us] +/- 66ms 2024-03-18T15:56:52.360 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +324us[ +324us] +/- 87ms 2024-03-18T15:56:52.360 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:56:52.381 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-18T15:56:52.382 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-03-18T15:56:52.382 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +2409us[+2409us] +/- 117ms 2024-03-18T15:56:52.382 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1438us[-1434us] +/- 66ms 2024-03-18T15:56:52.382 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +384us[ +384us] +/- 87ms 2024-03-18T15:56:52.382 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-18T15:56:52.399 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T15:56:52.409 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T15:56:52.410 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T15:56:52.468 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T15:56:52.548 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T15:56:52.620 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-18T15:56:52.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:56:52.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:52.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:56:52.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:52.777 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-18T15:56:52.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:52.807 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T15:56:52.824 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T15:56:52.877 INFO:teuthology.task.internal:Duration was 2063.921345 seconds 2024-03-18T15:56:52.877 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T15:56:52.926 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T15:56:52.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:56:52.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:56:52.931 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:56:52.970 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:56:52.971 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:56:52.991 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-18T15:56:53.482 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T15:56:53.482 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-03-18T15:56:53.482 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:56:53.503 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-03-18T15:56:53.503 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:56:53.529 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-03-18T15:56:53.529 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:56:53.554 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T15:56:53.554 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:56:53.556 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:56:53.572 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:56:53.622 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T15:56:53.622 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:56:53.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:56:53.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:56:54.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T15:56:54.017 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T15:56:54.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:56:54.043 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:56:54.071 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:56:54.112 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T15:56:54.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.126 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.128 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.148 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-03-18T15:56:54.150 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-03-18T15:56:54.184 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-03-18T15:56:54.206 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:54.220 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:54.234 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:56:54.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:56:54.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T15:56:54.271 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T15:56:54.272 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi096 2024-03-18T15:56:54.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:56:54.316 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi137 2024-03-18T15:56:54.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:56:54.353 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879/remote/smithi193 2024-03-18T15:56:54.353 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:56:54.391 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T15:56:54.391 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:56:54.394 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:56:54.396 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:56:54.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T15:56:54.453 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T15:56:54.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T15:56:54.464 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T15:56:54.464 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:56:54.467 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:56:54.469 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:56:54.478 INFO:teuthology.orchestra.run.smithi096.stdout: 265300 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:56 /home/ubuntu/cephtest 2024-03-18T15:56:54.481 INFO:teuthology.orchestra.run.smithi137.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:56 /home/ubuntu/cephtest 2024-03-18T15:56:54.494 INFO:teuthology.orchestra.run.smithi193.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:56 /home/ubuntu/cephtest 2024-03-18T15:56:54.495 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T15:56:54.507 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T15:56:54.591 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T15:56:54.617 DEBUG:teuthology.nuke:shortname: smithi096 2024-03-18T15:56:54.617 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:56:54.640 DEBUG:teuthology.nuke:shortname: smithi137 2024-03-18T15:56:54.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:56:54.645 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.941704', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKO40oG4Z608L2NhycTdYVxMm/Bb/T6zvFkPByFIkbZmmsLgb9YCPYUWiBfyBxOuWNxSX8BZkx/4DqXlYtw4eqw='} 2024-03-18T15:56:54.665 DEBUG:teuthology.nuke:shortname: smithi193 2024-03-18T15:56:54.665 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:56:54.668 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.943518', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGE9XDqXSfddSSffEy/B4ixPhL8iW/G9+DP92bxOlyu4CpzfZ7SvUSGdesZQLnkxyuvYfvbMS5zkfBRdz5QgFvs='} 2024-03-18T15:56:54.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608879', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:14:17.944708', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF7t/4WZwq6Lz36AVJ1CrMzvCrR9FpbE3jfiinrdJovPg2FhFVzpXdPntwY9RJBwXNcC7NCqpaY9ajBuNo5Muus='} 2024-03-18T15:56:54.702 INFO:teuthology.orchestra.console.smithi096:Power off 2024-03-18T15:56:54.702 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:56:54.725 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2024-03-18T15:56:54.725 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:54.743 INFO:teuthology.orchestra.console.smithi137:Power off 2024-03-18T15:56:54.743 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:56:54.753 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-18T15:56:54.767 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-03-18T15:56:54.767 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:54.782 INFO:teuthology.orchestra.console.smithi193:Power off 2024-03-18T15:56:54.783 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:56:54.793 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-18T15:56:54.804 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-03-18T15:56:54.804 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:54.825 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-03-18T15:56:58.754 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:58.794 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:58.826 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:56:58.864 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-03-18T15:56:58.904 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-18T15:56:58.936 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-03-18T15:57:02.866 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:57:02.906 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:57:02.937 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:57:02.979 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-03-18T15:57:03.014 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-18T15:57:03.045 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-03-18T15:57:03.079 INFO:teuthology.orchestra.console.smithi096:Power off completed 2024-03-18T15:57:03.115 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-03-18T15:57:03.146 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-03-18T15:57:03.252 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/blogbench}} duration: 2063.9213445186615 failure_reason: '"2024-03-18T15:40:03.632795+0000 mon.a (mon.0) 1140 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-18T15:57:03.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T15:57:03.375 INFO:teuthology.run:FAIL